Automated testing¶
GovReady-Q’s unit tests and integration tests are currently combined. Our integration tests uses Selenium to simulate user interactions with the interface.
To run the integration tests, you’ll also need to install chromedriver:
sudo apt-get install chromium-chromedriver (on Ubuntu)
brew cask install chromedriver (on Mac)
Navigate within your terminal to GovReady-Q top level directory.
Then run the test suite with:
python manage.py test
Note
Depending on your Python3 configuration, you may need to run
python3 manage.py test
To selectively run tests from individual modules:
# test rendering of guided modules
python manage.py test guidedmodules
# test general siteapp logic
python manage.py test siteapp
# test discussion functionality
python manage.py test discussion
Or to selectively run tests from individual classes or methods:
# run tests from individual test class
python manage.py test siteapp.tests.GeneralTests
# run tests from individual test method
python manage.py test siteapp.tests.GeneralTests.test_login
# run tests from different apps in sequence
python manage.py test siteapp.tests.GeneralTests.test_create_portfolios discussion.tests.DiscussionTests
To produce a code coverage report, run the tests with coverage:
coverage run --source='.' --branch manage.py test
coverage report
For Developers¶
If you want to run automated tests in visible mode you can set the parameter test_visible to true
in local/environment.json
otherwise tests will run in headless mode.
{
...
"test_visible": true,
...
}
if HEADLESS:
options.add_argument('--headless')
Selenium Troubleshooting¶
500 Internal Server Error
Receiving an 500 Internal Server Error in Selenium’s Chromium web browser during testing indicates an error serving the page.
If error is received only on some tests, the testing framework has located a legitimate problem rendering that page that needs to be corrected.
If the error occurs rendering every page, the probable cause is missing static files. Correct this problem
by re-fetch vendor resources, check your static
setting in the local/environment.json
file
and re-run Django collectstatic
admin command.
./fetch-vendor-resources.sh
python manage.py collectstatic
To debug further, set the verbosity of the tests to level 3 for increased log output and
look for Missing staticfiles manifest entry for
or other error messages detailing problems
with serving the page.
python manage.py test -v 3
IDE Test Configuration - Settings File Not Found
When executing tests through an IDE, the test configuration may error due to a lack of specifying a settings file. In the default test interpreter, set the settings file to siteapp/settings.py. This file contains the Django settings for the GovReady-Q project.
Windows(WSL) users¶
Aside from downloading (i.e. chromium-chromedriver) the executable to the system Windows needs a PATH that points to where the executable chromedriver file is located. This is also true for Windows Subsystem for Linux. A standard location to move the chromedriver executable to is /usr/local/bin. With that the program can just be named in your section of code without any other path parts needed. Below is an example of adding chromedriver as a parameter of the selenium chrome webdriver:
driver = selenium.webdriver.Chrome(executable_path='chromedriver.exe')
Note
The above applies for WSL 1 and for WSL 2 there are a few more steps to properly use chromedriver. If you ever want to set your Ubuntu install to use a different WSL version then in a windows terminal run wsl --set-version Ubuntu <new_version_number>
WSL version 2¶
For WSL 2 on Ubuntu you need to do these steps even if you have Chrome installed in Windows.
Dependencies:
sudo apt-get update
sudo apt-get install -y curl unzip xvfb libxi6 libgconf-2-4
Chrome itself:
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo apt install ./google-chrome-stable_current_amd64.deb
Ensure it worked:
google-chrome --version
Find the chromedriver url for the ChromeDriver version that matches your Chrome version (e.g. https://chromedriver.storage.googleapis.com/86.0.4240.22/chromedriver_linux64.zip)
Download, unzip, and put in your local bin directory:
wget https://chromedriver.storage.googleapis.com/86.0.4240.22/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
sudo mv chromedriver /usr/local/bin/chromedriver
sudo chown root:root /usr/bin/chromedriver
sudo chmod +x /usr/local/bin/chromedriver
chromedriver should now point to the newly installed chromedriver
which chromedriver # /usr/local/bin/chromedriver
Last but not least we need to download and install VcXsrv. Then run xlaunch.exe from the programs files folder (for VcXsrv). Leave most settings as default but check the “Disable access control”. In Linux the DISPLAY environment variable tells GUI applications at which IP address the X Server is that we want to use. Since in WSL 2 the IP address of Windows land is not localhost
anymore, we need to set DISPLAY to the correct IP address:
export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2; exit;}'):0.0
Note
You can put it anywhere but I recommend .bashrc.
Now if you run echo $DISPLAY
you should get something like 172.17.35.177:0.0
.