When you write automated tests, it’s important to make sure that those tests can fail. This can be done by mutating the test so it’s expected conditions are not met, so that the test will fail (not error). When you satisfy those conditions by correcting the inputs to the test, you can have more confidence that your test is actually testing what you think it is — or at least that it is testing something.
It’s easy to make a test fail, and then change it to make it pass , but testing your tests can be more complex than that — which is a good reason why tests should be simple. You’ll only catch the error that you explicitly tested for (or completely bogus tests like assert true == true.)
Not to say those simple sanity checks don’t have value, but an even better check is to write a test that fails before a change, but passes after the change is applied to the system under test. This is easier to do with unit tests, but for system tests there is great value in seeing tests fail before a feature (or bug fix) is deployed and then seeing it succeed afterwards.
It can still lead to bogus tests (or at least partially bogus tests) but a few of these type of tests being run after a deployment are extremely valuable and can catch all kinds of issues, as well as give greater confidence that what you added actually worked. This is especially useful when moving code (and configuration) through a delivery pipeline across multiple environments—from dev to test to stage to production.
Having (and tracking) these sort of tests — which pass only when the change is applied makes the delivery pipeline much more valuable,
Also don’t forget the other tests — those that make sure what you changed didn’t break anything else — although these are the more common types of automated regression tests.
Originally posted in response to Bas Dijkstra on LinkedIn:
Selenium is a programming tool used to automate the browser — simulating a real user by opening windows, going to URLs, clicking buttons, filling in forms, etc. Selenium is primarily used for test automation, but can also be used for other things.
It is also referred to as Selenium WebDriver, because there were two different projects (Selenium & WebDriver) which did similar things and eventually merged. Selenium uses the WebDriver protocol (a W3C standard now) to communicate over the network via a REST API. This allows for remote automation,
There are other tools associated with Selenium, including Selenium Grid — which enables remote execution of automation and Selenium IDE — which allows you to record and play back automated steps without writing code, and has (limited) ability to to export from Selenium IDE to code that can run independent.
Selenium IDE does not support multiple users, but scripts can be exported and shared from one user to another.
Selenium Grid allows for parallel remote execution of Selenium scripts, which allows multiple people (or tests) to execute tests at the same time.
The concept of “supporting multiple users” does not really make sense in terms of Selenium as an open source development tool or coding library.
It would be like saying: “Does Microsoft Word support multiple users?” or “Does the Java programming language support multiple users?”
In the case of Microsoft Word, every user that has the program can use it, but collaboration is (primarily) done outside of the tool. With proprietary software like Microsoft Word, each user may need a license to run their own copy of the application, but Selenium is open source, so does not require the purchase of any license to use.
And as a programming library, any number of users can reference Selenium in their own code and execute it. Users can run multiple automated tests (or other scripts) at once — if they write their program to run in parallel.
But in order to maximize parallel execution (for single or multiple users) you need to have a remote Selenium grid. There is an open source grid that anyone can deploy, but there are also commercial services the host Selenium Grid with additional tools at a cost. These companies include Sauce Labs, BrowserStack, LambdaTest, and Applitools. Each company has their own policy about multiple users, and of the ones I mentioned, they all support multiple users in their own way.
No, you can’t export a table from HTML to Excel or MySQL using Selenium with Python.
But you’re in luck! You just asked the wrong question.
It’s like if you had asked: “Can I build a bookshelf using my DeWalt Table Saw?”
The answer is, of course, still no, but a reasonable person would say that your table saw is one of the tools you could use, but you’re also going to need a hammer, and some nails — or a screwdriver and screws — and maybe some paint or varnish. And a couple clamps to hold it all to together while you’re building it.
Unfortunately, I am not a reasonable person. But I’ll show you how to do it anyway.
You can use Selenium with Python to extract content from an HTML table. You can then use other tools (with Python) to import that content into an Excel spreadsheet or MySQL database.
For example, I’ll fetch cryptocurrency prices from CoinMarketCap using Selenium:
# get the HTML table data using Selenium
from selenium import webdriver
url = "https://coinmarketcap.com/"
table_locator = "xpath", "//table"
driver = webdriver.Chrome()
driver.get(url)
table_webelement = driver.find_element(*table_locator)
table_html = table_webelement.get_attribute("outerHTML")
In addition to Selenium, I’d recommend using Pandas DataFrames to export to Excel — both because it’s easier than working with openpyxl directory (or xlwt, or xlsxwriter, or one of several other lower level libraries) and because pandas has all sorts of other great data manipulation features that might come in handy. (And it looks great on a resume!)
Here’s how you can use Python to read an HTML table directly into a Pandas Dataframe and then export it to a Microsoft Excel Spreadsheet using DataFrame.to_excel()
# load the HTML table to Pandas DataFrame
import pandas
dataframes = pandas.read_html(table_html)
# get the first and only table on the page
table_dataframe = dataframes[0]
# export data to Excel
table_dataframe.to_excel("data.xlsx")
Here is the resulting spreadsheet:
Or you can export the Dataframe to a database.
Here, we use MySQL Connector with SQL Alchemy to append our results from the HTML table to a table named “prices” into the MariaDB “test” database. If the table does not exist, it creates it.
# export data to MySQL (or MariaDB)
import sqlalchemy
from mysql import connector
conn = sqlalchemy.create_engine("mysql+mysqlconnector://username:password@localhost/test")
table_dataframe.to_sql(con=conn, name="prices", if_exists='append', index=False)
And our resulting table will look like this:
There are also other tools you may be able to use:
requests HTTP library instead of Selenium if you don’t need to manipulate the browser to fetch data
# get prices using requests
import requests
response = requests.get("https://coinmarketcap.com/")
# parse data using Beauitful Soup
from bs4 import BeautifulSoup
soup = BeautifulSoup(response.content, "html.parser")
table_soup = soup.find("table")
headings = [th.text.strip() for th in table_soup.find_all("th")]
rows = []
for tr in table_soup.find_all('tr', limit=11):
row = []
for td in tr.find_all('td'):
row.append(td.text.strip())
rows.append(row)
csv to parse into a CSV instead of Excel format (a CSV file can also be loaded directly into MySQL without Pandas).
#export data to csv file
import csv
with open("data.csv", mode="w") as csvfile:
writer = csv.writer(csvfile)
writer.writerow(headings)
writer.writerows(rows)
Of course , you can also load your data List into pandas as well:
table_dataframe = pandas.DataFrame(rows)
Here is a gist showing all the code together:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
which take as an argument a string representing the JavaScript code you want to execute on the browser and (optionally) one or more arguments. If the second argument is a WebElement it will apply the script to the corresponding HTML element. Arguments are added to the JS magic arguments variable which represents the values passed to a function. If the code executed returns a value, that is returned to your Selenium code
Each driver is responsible for implementing it for the browser.
RemoteWebDriver implements it as well.
But when *you* as a Selenium user want to use JavaScriptExecutor is when you assign a driver to the base type WebDriver, which does not implement it.
in this case, you cast your driver instance (which really does implement executeScript() and executeScriptAsync().
For example
WebDriver driver = new ChromeDriver();
// base type ‘WebDriver’ does not define executeScript() although our instance that extends RemoteWebDriver actually does implement it.
// So we need to cast it to ‘JavaScriptExecutor’ to let the Java compiler know.
// this time we use native JavaScript on the browser to find an element and return its HTML, bypassing Selenium’s ability to do so.
The above two examples illustrate ways you can accomplish in JavaScript what you would normally use Selenium for.
Why would you do this?
Well, sometimes the driver has a bug, or it can be more efficient (or reliable) to do in JavaScript, or you might want to combine multiple actions in 1 WebDriver call.
If you have access to your crontab you can set a Selenium script to run periodically. If you don’t have cron, you can use a VM (with Vagrant) or Container (with Docker) to get it.
Cron is available on Linux & Unix systems. On Windows, you can use Task Scheduler. On Mac, there is launchd, but it also includes cron (which wraps launchd).
You could also set up a job to run on a schedule using a continuous integration server such as Jenkins. Or write a simple, long running script that runs in the background and sleeps between executions.
I have a service that runs Selenium tests and monitoring for my clients, and use both cron and Jenkins for executing test runs regularly. I also have event-triggered tasks that can be triggered by a checkin or user request.
Each line represents a task with schedule in the following format:
#minute #hour #day #month #weekday #command
# perform a task every weekday morning at 7am
* 7 * * 1-5 wakeup.sh
# perform a task every hour
@hourly python selenium-monitor.py
You can edit crontab to create a task by typing crontab -e
You can view your crontab by typing crontab -l
If you just want to repeat your task within your script while it’s running, you can add a sleep statement and loop (either over an interval or until you kill the script).
#!/usr/bin/env python
from time import sleep
from selenium import webdriver
sites = ['https://google.com', 'https://bing.com', 'https://duck.com']
interval = 60 #seconds
iterations = 10 #times
def poll_site(url):
driver = webdriver.Chrome()
driver.get(url)
title = driver.title
driver.quit()
return title
while (iterations > 0):
for url in sites:
print(poll_site(url))
sleep(interval)
iterations -= 1
See the example code on github:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I’m working on a test framework that needs to be run by less-technical testers. The tests are data driven from a spreadsheet (google docs spreadsheet API + gdata.)
Tests will be run locally (for now at least) since there isn’t a test lab available for remote execution, and no CI. I didn’t want to have to require users to install NUnit to execute tests.
At first I started by writing a main() class and rolling my own assertions. But I decided that the parameterized test features of NUnit were worth the effort of a little research. NUnit can, in fact, be run programmatically, though the execution appears less flexible than with other frameworks.
I created a TestRunner class with main function:
using System;
using NUnit.Core;
using NUnit.Framework.Constraints;
using NLog;
namespace oneshore.qa.testrunner
{
class TestRunner
{
Logger log = NLog.LogManager.GetCurrentClassLogger();
public static void Main(String[] args)
{
//get from command line args
String pathToTestLibrary = "C:\\dev\\oneshore.Tests.DLL";
DateTime startTime = System.DateTime.Now;
log.Info("starting test execution...");
TestRunner runner = new TestRunner();
runner.run(pathToTestLibrary);
log.Info("...test execution finished");
DateTime finishTime = System.DateTime.Now;
TimeSpan elapsedTime = finishTime.Subtract(startTime);
log.Info("total elapsed time: " + elapsedTime);
}
public void run(String pathToTestLibrary)
{
CoreExtensions.Host.InitializeService();
TestPackage testPackage = new TestPackage(@pathToTestLibrary);
testPackage.BasePath = Path.GetDirectoryName(pathToTestLibrary);
TestSuiteBuilder builder = new TestSuiteBuilder();
TestSuite suite = builder.Build(testPackage);
TestResult result = suite.Run(new NullListener(), TestFilter.Empty);
log.Debug("has results? " + result.HasResults);
log.Debug("results count: " + result.Results.Count);
log.Debug("success? " + result.IsSuccess);
}
}
}
The advantage to running tests this way is that NUnit does not need to be installed (though DLLs for NUnit — nunit.core.dll & nunit.core.interfaces.dll — and any other dependencies like NLog need to be packaged with the TestRunner.) You can still write and execute your tests from NUnit while under development.
One disadvantage is that you don’t have the full test results by using the TestSuiteBuilder to bundle every test it finds into one suite. I’d like to find a way to improve that. You also can’t run more than one test assembly at the same time — you can create a nunit project xml for that — and at that point you might as well bundle the nunit test framework.
For now, my base test class (that each of my Nunit tests inherit from) reports via catching and counting assertion failures and writing to a log file. It can then use the Quality Center integration tool I described in an earlier post, though I’m planning on wiring it all together eventually, so anyone can run tests automatically by clicking on an icon, using a File Picker dialog to select the test library (see upcoming post) and have test results entered in QC.
This will allow distributed parameterized testing that can be done by anyone. I may try to set up a web UI like fitnesse for data driven tests as well.
Fijiaaron 