To shorten the development cycle of your Web application you need to start testing it on the early stages of the project. It seems obvious, but many enterprise IT organizations haven’t adopted agile testing methodologies, which costs them dearly. JavaScript is dynamically typed interpreted language - there is no compiler to help in identifying errors as it’s done in compiled languages like Java. This means that a lot more time should be allocated for testing for JavaScript Web applications. Moreover, a programmer who didn’t introduce testing techniques into his daily routine can’t be 100% sure that his code works properly.
The static code analysis and code quality tools such as Esprima and JSHint will help in reducing the number of syntax errors and improve quality of your code.
|
Tip
|
We’ve demonstrated how to setup JSHint for your JavaScript project and automate the process of checking your code for the syntax errors in a chapter «Selected Productivity Tools for Enterprise Developers». |
To switch to a test-driven development mode, make testing a part of your development process in its early stages rather than scheduling testing after the development cycle is complete.
Introduction of test-driven development may substantially improve your code quality. It is very important to receive the feedback about your code on a regular basis. That’s why tests must be automated and should run as soon as you’ve changed the code.
There are many testing frameworks in JavaScript world, but we’ll give you a brief overview of two of them: QUnit and Jasmine. The main goal of each framework is to test small pieces of code a.k.a. units.
We will go through basic testing techniques known as "Test-Driven Development" and "Test First". You’ll learn how to automate the testing process in multiple browsers with Testem Runner or by running tests in so called headless mode with PhantomJS.
The second part of this chapter is dedicated to setting up a new Save The Child project in the IDE with selected test frameworks.
Any software has bugs. But in interpreted languages like JavaScript you don’t have help of compilers that could have pointed you at the potential issues on early stages of development. You need to continue testing code over and over again to catch regression errors, to be able to add new features without breaking the existing ones. Code that is covered with tests is easy to refactor. Tests help to prove correctness of your code. A well tested code leads to better overall design of your programs.
In this chapter we’ll discuss the following types of testing:
-
Unit testing
-
Integration testing
-
Functional testing
-
Load (a.k.a. stress) testing
Although, Quality Assurance (QA) and User Acceptance Testing (UAT) is far beyond the scope of this chapter you need understand the difference.
Software QA (or Quality Control (QC)) is the process that helps identify the correctness, completeness, security compliance and the quality of the software. QA testing is performed by the specialists (testers, analysts). The goal of QA testing is to ensure that the application complies with a set of the predefined behavior requirements.
UAT is performed by business users or subject area experts. The UAT should result in the endorsement that the tested applications/functionality/module meets the agreed upon requirements. The results of UAT gives the confidence to the end-user that the system will perform in production according to specification.
During the QA process the specialist intents to perform all tests trying to break the application. This approach helps finding the errors undiscovered by developers. On the contrary, during UAT the user runs business-as-usual scenarios and makes sure that business functions are implemented in the application.
Let’s go over the strategies, approaches, and tools that will help you in test automation.
The unit test is a piece of code that invokes a method being tested. It asserts some assumptions about the application logic and behavior of the method. Typically you’ll be writing such tests using unit-testing framework of your choice. Tests should run fast, be automated with clear output. For example you can test if a function is called with particular arguments, it should return expected result. We will take a closer look on unit testing terminology and vocabulary in a Test Driven Development section.
Integration testing is a phase when already tested units are combined into a module to test the interfaces between them. You may want to test the integration of your code with the code written by other developers, e.g. some third-party framework. Integration tests ensure that any abstraction layers we build over the third-party code work as expected. Both unit and integration tests are written by application developers.
Functional testing is aimed at finding out whether the application properly implements business logic. For example, if the user clicks on a row in the grid with customers, the program should display a form view with specific details about the selected customer. In functional testing business users should define what has to be tested, unlike unit or integration testing where tests are created by software developers.
Functional tests can be performed manually by a real person clicking through each and every view of the web application confirming that it operates properly or reporting discrepancies with the functional specifications.
But there are tools to automate the process of functional testing of Web applications. Such tools allow to record the users' actions and replay them in the automatic mode. Below are brief descriptions of two of such tools - Selenium and CasperJS.
-
Selenium is an advanced browser automation tool suite that has capabilities to run and record user scenarios without requiring developers to learn any scripting languages. Also Selenium has an API for integration with many programming languages like Java, C#, JavaScript and etc. Selenium uses the WebDriver API to talk to the browsers and receive running context information. WebDriver is becoming the standard API for browser automation. Selenium supports a wide range of browsers and platforms.
-
CasperJS is a scripting framework written in JavaScript. CasperJS allows to create interaction scenarios like defining and ordering the navigation steps, filling and submitting forms or even scrapping Web content and making Web page screenshots. CasperJS works on top of PhantomJS and SlimerJS browser, which limits the testing runtime environment to WebKit-based and Gecko-based browsers. Still it’s a very useful tool when you want to run tests in a Continuous Integration (CI)environment. You can read more about PhantomJS and SlimerJS and CI in the corresponding sidebars later in this chapter.
PhantomJS is a headless WebKit-based rendering engine and interpretation with JavaScript API. Think of PhantomJS as a browser that doesn’t have any graphical user interface. PhantomJS can execute HTML, CSS, and JavaScript code. Because PhantomJS is not required to render browsers’s GUI, it can be used in display-less environments (e.g. CI server) to run tests. SlimerJS follows the same idea of headless browser, similar to PhantomJS, but it uses Gecko engine instead.
PhantomsJS is built on top of Webkit and JavaScriptCore (like Safari) and SlimerJS is built on top of Gecko and SpiderMonkey (like Firefox). You can find the comprehensive list of difference between PhantonJS and SlimerJS APIs in SlimerJS’s documentation site.
In our case, Grunt automatically spawns the PhantomJS instance, executes the code of our tests, reads the execution results using PhantomJS API, and prints them out in the console. If you’re not familiar with Grunt tasks, please, refer to [appendix_a] for additional information about usage Grunt in our Save The Child project.
Load testing is a process that can help in answering the following questions:
-
How many concurrent users can work with your application without bringing your server to its knees?
-
Even if your server is capable of serving a thousand users, is your application performance in a compliance with the Service Level Agreement (SLA), if any?
It all comes down to two factors: availability and response time of your application. Ideally, these requirements should be well defined in the SLA document, which should clearly state what metrics are acceptable from the user’s perspective. For example, the SLA can include a clause stating that the initial download of your application shouldn’t take longer than 10 seconds for users with a slow connections (under 1Mbps). SLA can also state that the query to display a list of customers shouldn’t run for more than five seconds, and the application should be operational 99.9 percent of the time.
To avoid surprises after going live with your new mission-critical web application, don’t forget to include in your project plan an item to create and run a set of heavy stress tests. Do this well in advance before your project goes live. With load testing, you don’t need to hire a thousand of interns to play the roles of concurrent users to find out whether your application will meet the SLA.
Automated load testing software allows you to emulate required number of users, set up the throttling to emulate a slower connection, and configure the ramp-up speed. For example, you can simulate a situation where the number of users logged on to your system grows at the speed of 50 users every 10 seconds. Stress testing software also allows you to pre-record the users interactions, and then you can run these scripts emulating a heavy load.
Professional stress-testing software allows simulating the load close to the real-world usage patterns. You should be able to create and run mixed scripts simulating a situation in which some users are logging on to your application while others are retrieving the data and performing data modifications. Below are some tools worth considering for load testing.
-
Apache Benchmark is a simple to use command line tool. For example, with a command
ab -n 10 -c 10 -t 60 http://savesickchild.org:8080/ssc_extjs/Apache Benchmark will open 10 concurrent connection with the server and will send 10 requests via each connection to simulate 10 visitors working with your Web application during 60 seconds. The number of concurrent connections is the actual number of concurrent users. You can find an Apache Benchmark sample report in following snippet.
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Server Software: GlassFish
Server Hostname: savesickchild.org
Server Port: 8080
Document Path: /ssc_extjs/
Document Length: 306 bytes
Concurrency Level: 10
Time taken for tests: 60.003 seconds
Complete requests: 17526
Failed requests: 0
Total transferred: 11988468 bytes
HTML transferred: 5363262 bytes
Requests per second: 292086.73
Transfer rate: 199798.72 kb/s received
Connnection Times (ms)
min avg max
Connect: 10 13 1305
Processing: 11 14 12
Total: 21 27 1317-
Apache JMeter is a tool with a graphic user interface. It can be used to simulate heavy load on a server, network or an object to test its strength or to analyze overall performance under different load types. You can find more about testing web applications using JMeter in official documentation.
-
Please, refer the sidebar called What is PhantomJS to make yourself familiar with this tool. The slide deck titled "Browser Performance metering with PhantomJS" is yet another good resource for seeing how PhantomJS can be used for performance testing.
The methodology known as Test-Driven Development (TDD) substantially changes the way a traditional software development is done. This methodology wants you to write tests even before writing the application code. Instead of just using testing to verify our work after it’s done, TDD moves the testing into the earlier application design phase. You should use the tests to clarify your ideas about what you are about to program. Here is fundamental mantra of TDD:
-
Write a test and make it fail.
-
Make the test pass.
-
Refactor.
-
Repeat.
This technique also referred as "Red-Green-Refactoring" because IDE’s and test runners use red color to indicate failed tests and green color to indicate the tests that passed.
When you are about to start programming a class with some business logic, ask yourself, "How can I ensure that this function works as expected?" After you know the answer, write a test JavaScript class that calls this function to assert that the business logic gives the expected result.
An assertion is a true-false statement that represents what a programmer assumes about program state, e.g. customerID >0 is an assertion.
According to 'Martin Fowler', an assertion is a section of code that works only if certain conditions are true. It’s a conditional statement that is assumed to be always true. Failure of an assertion results in test failure.
Run your test, and it will immediately fail because no application code is written yet! Only after the test is written, start programming the business logic of your application
You should write a simplest possible piece of code to make the test pass. Don’t try to find a generic solution at this step. For example, if you want to test a calculator that needs to return 4 as result of 2+2 write the code what simply returns 4. Don’t worry about the performance or optimization at this point in time. Just make the test pass. Once you made it, you can refactor your application code to make it more efficient. Now you might want to introduce a real algorithm for implementing the application logic without worrying about breaking the contract with other components of your application.
A failed unit test indicates that your code change introduced regression, which is a new bug in a previously worked software. Automated testing and well-written test cases can reduce the likelihood of a regression in your code.
TDD allows to receive feedback from your code almost immediately. It’s better to find that something is broken during development rather than in the application deployed in production.
|
Note
|
Learn by heart The Golden Rule Of TDD:
|
In addition to business logic, web applications should be tested for proper rendering of UI components, changing view states, dispatching, and handling events.
With any testing framework, your tests will follow same basic pattern. First, you need to setup up the test environment. Second, you run the production code and check that it works as it supposed to. Finally, you need to clean up after the test will run - remove everything that your program has created during setup of the environment.
This pattern for authoring unit tests is called Arrange-Act-Assert-Reset (AAAR [1]).
-
In the Arrange phase you set up the unit of work to test. For example, create a DOM element.
-
In the Act phase you exercise the unit under test and capture the resulting state. You execute your production code in unit test context.
-
In the Assert phase you verify the behavior through assertions.
-
In the Reset phase, you reset the environment to the initial state. For example, erase the DOM elements created in the Arrange phase. Most of the frameworks provide a "teardown" function that would be invoked after test is done.
Later in this chapter, you’ll see how different frameworks implements AAAR pattern.
In next sections we will dive into the testing frameworks for JavaScript.
We’ll start our journey to JavaScript testing frameworks with QUnit, which was originally developed by John Resig as part of jQuery. QUnit now runs completely standalone and doesn’t have any jQuery dependencies. While it’s still being used by the jQuery Project itself for testing jQuery, jQuery UI and jQuery Mobile code, QUnit can be used to test any generic JavaScript code.
In this section you’re going learn how to automatically run the Qunit tests using Grunt. Let’s setup our project by adding the Qunit framework and tests file. Start with downloading the latest version from QUint Web site. You need to get only two files - qunit-.js and qunit-.css.
'use strict';
test("my first qunit test", function () {
ok(2 + 2 === 4, "Passed!");
});You’ll also need a test runner for the test setup. A test runner is an html file contains links to QUnit framework JavaScript file.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>QUnit Example</title>
<link rel="stylesheet" href="lib/qunit-1.11/qunit.css">
<script src="lib/qunit-1.11/qunit.js"></script>
<script type="text/javascript" src="../app/js/libs/jquery-1.9.0.min.js"></script> <!--(1)-->
<script src="test/tests.js"></script> <!--(2)-->
</head>
<body>
<div id="qunit"></div> <!--(3)-->
<div id='qunit-fixture'>
<!--(4)-->
</div>
</body>
</html>-
In this section we continue working on the jQuery-based version of the Save The Child application. Hence our "production environment" depends on availability of jQuery, so we need to include jQuery in the test runner.
-
Test files are included too.
-
QUnit fills this block with results.
-
Any HTML you want to be present in each test. It will be reset for each test.
To run all our tests we need to open the qunit-runner.html in browser. (See A test run results in browser).
module.exports = function (grunt) {
"use strict";
grunt.initConfig({
qunit: {
all: ["test/qunit-runner.html"]
}
});
grunt.registerTask("test", "qunit");
grunt.loadNpmTasks("grunt-contrib-qunit"); // (1)
};-
Grunt loads task from local NPM repository. To install this tasks in node_modules directory use command 'npm install grunt-contrib-qunit'.
Now let’s briefly review QUnit API components. You can find a typical QUnit script in the following listing.
(function($) {
module('SaveSickChild: login component test', { // (1)
setup: function() { // (2)
// test setup code goes here
},
teardown: function() { // (3)
// test cleanup code goes here
}
});
test('jquery is here', function() { // (4)
ok($, "yes, it's here");
});
test("2 add 2 equals 4", function() {
ok(2 + 2 === 4, "Passed!"); // (5)
});
test('2 add 2 not equals 5', function() {
notEqual(2 + 2, 5, "failed"); // (6)
});
}(jQuery)); // (7)-
A
modulefunction allows to combine related tests as group. -
Here we can run Arrange phase. A
setupfunction will be called before each test. -
A
teardownfunction will be call after each test respectively. This is our Reset phase. -
You need to place code of your test in corresponding
testfunction. -
Typically, you need to use assertions to make sure the code being tested gives expected results. The function
okwill examine the first argument to betrue. -
A pair of functions
equalandnotEqualwill check for the equivalence of the first and second arguments, which could be expressions as well. -
A code of the test is wrapped in immediate-invoked function expression (IIFE) and passes
jQueryobject as$variable.
You can find more details about QUnit in product documentation and QUnit Cookbook.
The idea behind behavior-driven development (BDD) is to use the natural language constructs to describe what you think your code should be doing or more specifically, what your functions should be returning.
Similarly to unit tests, with BDD you write short specifications that test one feature at a time. Specifications should be sentences. For example, "Calculator adds two positive numbers". Such sentences will help you to easy identify the failed test by simply reading this sentence in the resulting report.
Now we’ll demonstrate this concept using Jasmine - the BDD frameworks for JavaScript. Jasmine provides a very nice way to group, execute, and report JavaScript unit tests.
You’ve learned already how to use the Grunt tool to automate boilerplate tasks. Now let’s execute a Jasmine specification with Grunt. The next section covers Jasmine basics, but for now think of Jasmine as a piece of code that should be executed by Grunt.
Let’s start with downloading the latest version of Jasmine from its github repository. Unzip the archive. The content of unzipped folder will look similar to the screenshot Jasmine Distribution.
Jasmine comes with an example spec (spec folder) and an html test runner - SpecRunner.html. Let’s open the file SpecRunner.html in browser Running Jasmine Specs in a Browser.
The SpecRunner.html is structured similarly to QUnit html runner. You can run specifications by opening the runner file in browser.
link:include/SpecRunner.html[role=include]-
Required Jasmine framework library.
-
Include the source files.
-
Include the specification code. It’s not required but files that contain specification code can have suffix
*Spec.js. -
Initialize Jasmine and run all specifications when the page is loaded.
Now let’s update the Grunfile to run the same sample specifications with the PhantomJS headless browser. Copy the content of src folder of your Jasmine distribution into the app/js folder of our project, and then copy the content of the spec folder into the test/spec folder of your project. Also create a folder test/lib/jasmine and copy the content of Jasmine distribution lib folder there. Jasmine Specifications in Our Project
Now you need to edit Gruntfile_jasmine.js to activate Jasmine support.
module.exports = function (grunt) {
'use strict';
grunt.initConfig({
jasmine: { // (1)
src: ['app/js/Player.js', 'app/js/Song.js'], // (2)
options: {
specs: 'test/spec/PlayerSpec.js', // (3)
helpers: 'test/spec/SpecHelper.js' // (4)
}
}
});
// Alias the `test` task
grunt.registerTask('test', 'jasmine');
// loading jasmine grunt module
grunt.loadNpmTasks('grunt-contrib-jasmine'); // (5)
};-
Configuring the Jasmine task.
-
Specifying the location of the source files.
-
Specifying the location of Jasmine specs.
-
Specifying the location of Jasmine helpers, which will be covered later in this chapter.
-
Grunt loads task from local NPM repository. To install this tasks in node_modules directory use command 'npm install grunt-contrib-jasmine'.
To execute tests, run the command grunt --gruntfile Gruntfile_jasmine.js jasmine, and you should see something like this:
Running "jasmine:src" (jasmine) task
Testing jasmine specs via phantom
.....
5 specs in 0.003s.
>> 0 failures
Done, without errors.In this example, Grunt successfully executed the tests with PhantomJS of all five specifications defined in PlayerSpec.js.
Continuous Integration (CI) is a software development practice where members of a team integrate their work frequently, which results in multiple integrations per day.
Introduced by Martin Fowler and Matthew Foemmel, the theory of continuous integration [2] recommends creating scripts and running automated builds (including tests) of your application at least once a day. This allows you to identify issues in the code early.
Authors of this book successfully use an open source framework called Jenkins (there are other similar CI servers) for establishing continuous build process.
With Jenkins, you can have scripts that run either at a specified time interval or on each source code repository check-in of the new code. You may also force an additional build process whenever you like. The Grunt command line tool should be installed and be available on CI machine to allow the Jenkins server invoke Grunt scrips and publish test results.
We use it to ensure continuous builds of the internal and open source projects.
In the next section you will learn how write your own specifications.
After we’ve set up the tools for running tests, let’s start developing tests and learn the Jasmine framework constructs. Every specification file has a set of suites defined in the describe function. Suites help logically organize code of test specifications.
describe("My function under test should", function () { // (1)
it("return on", function () { // (2)
// place specification code here
//
});
describe("another suite", function () { // (3)
it("spec1", function () {
});
});
it("my another spec", function () { // (4)
var truth = true;
expect(truth).toBeTruthy();
});
it("2+2 = 4", function () {
expect(2 + 2).toEqual(4); // (5)
});
});-
The function
describe()accepts two parameters - the name of the test suite, and the callback function. The function is a block of code that implements the suite. If for some reasons you would like to skip the suite from execution you can just use methodxdescribe()and a whole suite will be excluded until you rename it back todescribe(). -
The function
it()also accepts similar parameters - the name of the test specification, and the function that implements this specification. Like in case with suites, Jasmine has a correspondingxitmethod to exclude specification from execution. -
Each suite can have any number of nested suites.
-
Each suite can have any number of specifications.
-
The code checks to see if
2+2equals4. We used the functiontoEqual(), which is a matcher. Define expectations with the functionexpect(), which takes a value, called the actual. It’s chained with a matcher function, which takes the expected value (in our case it’s 4) and checks if it satisfies the criterion defined in the matcher.
Various flavors of matchers are shipped with Jasmine framework, and we’re going to review a couple the frequently used matchers functions.
-
Equality
Function
toEqual()checks if two things are equal. -
True or False?
Functions
toBeTruthy()andtoBeFalsy()checks if something is true or false respectively. -
Identity
Function
toBe()checks if two things are the same object. -
Nullness
Function
toBeNull()checks if something isnull. -
Is Element Present
Function
toContain()check if an actual value is an element of array.expect(["James Bond", "Austin Powers", "Jack Reacher", "Duck"]).toContain("Duck");
-
Negate Other Matchers
This function is used to reverse matchers to ensure that they aren’t
true. To do that, simply prefix things with.not:expect(["James Bond", "Austin Powers", "Jack Reacher"]).not.toContain("Duck");
Above we’ve listed only some of existing matchers. You can find the complete documentation with code examples at official Jasmine website and at wiki.
|
Tip
|
There is a large set of jquery-specific matchers available https://github.com/velesin/jasmine-jquery |
Jasmine framework has an API to arrange your specification (based on [AAAR] concept). It includes two methods - beforeEach() and afterEach(), which allow you to execute some code before and after each spec respectively. It’s very useful for instantiation of the shared objects or cleaning up after the tests complete.
If you need to fulfill your test with some common dependencies or setup the environment, just place code inside beforeEach() method. Such dependencies and environment are known as fixture.
What is Fixture?
Test fixture refers to the fixed state used as a baseline for running tests. The main purpose of a test fixture is to ensure that there is a well known and fixed environment in which tests are run so that results are repeatable. Sometimes a fixture also referred as test context.
(function($) {
describe("DOM manipulation spec", function() {
var usernameInput;
var passwordInput;
beforeEach(function() { // (1)
usernameInput = document.createElement("input"); // (2)
usernameInput.setAttribute("type", "text");
usernameInput.setAttribute("id", "username");
usernameInput.setAttribute("name", "username");
usernameInput.setAttribute("placeholder", "username");
usernameInput.setAttribute("autocomplete", "off");
passwordInput = document.createElement("input");
passwordInput.setAttribute("type", "text");
passwordInput.setAttribute("id", "password");
passwordInput.setAttribute("name", "password");
passwordInput.setAttribute("placeholder", "password");
passwordInput.setAttribute("autocomplete", "off");
});
afterEach(function () { // (3)
});
it("jquey should be present", function() {
expect($).not.toBeNull();
});
it("inputs should exist", function() {
expect(usernameInput.id).toBe("username");
expect(passwordInput.id).toBe("password");
});
it("should not allow login with empty username and password and return code equals 0", function() {
var result = ssc.login(usernameInput, passwordInput); // (4)
expect(result).toBe(0);
});
it("should allow login with user admin and password 1234 and return code equals 1", function() {
usernameInput.value = "admin"; // (5)
passwordInput.value = "1234";
var result = ssc.login(usernameInput, passwordInput);
expect(result).toBe(1);
});
});
})(jQuery);-
This method will be called before each specification.
-
In the
beforeEach()method we create two input fields. These two inputs will be available in all specifications of this suite. -
You can place additional cleanup code inside
afterEach()function. -
A
beforeEach()function helps to implement Don’t Repeat Yourself principle in our tests. You don’t need to create the dependency elements inside each specification manually. -
You can change defaults inside each specification without worrying about affecting other specifications. Your test environment will be reset for each specification.
Jasmine framework is easily extendable, and it allows you to define your own matchers if for some reasons you’re unable to find the appropriate matchers in the Jasmine distribution. In such cases you’d need to write a custom matcher. Let’s write a matcher that check if a string contains name of the "secret agent" from the defined list of agents.
toBeSecretAgent matcherbeforeEach(function () {
this.addMatchers({
toBeSecretAgent: function () {
var agentList = [
"James Bond",
"Ethan Hunt",
"Jason Bourne",
"Aaron Cross",
"Jack Reacher"
];
var actual = this.actual; // (1)
this.message = function () { // (2)
return actual + " is not a secret agent";
};
return agentList.indexOf(actual) !== -1; // (3)
}
});
});-
this.actualcontains the value used as arguments in theexpectfunction. -
We can customize error message if the test fails.
-
This function checks if
agentsListcontains the actual value.
The invocations of this helper can look like this:
it("part of super agents", function () {
expect("James Bond").toBeSecretAgent(); // (1)
expect("Jason Bourne").toBeSecretAgent();
expect("Austin Powers").not.toBeSecretAgent(); // (2)
expect("Austin Powers").toBeSecretAgent(); // (3)
});-
Calling the custom matcher.
-
Custom matchers could be used together with the
.notmodifier. -
This expectation will fail because 'Austin Powers' is not in the list of secret agents.
The following custom failure message will be displayed on the console.
/usr/local/share/npm/bin/grunt --gruntfile Gruntfile_jasmine.js jasmine
Running "jasmine:src" (jasmine) task
Testing jasmine specs via phantom
...........x
My function under test should:: part of super agents: failed
"Austin Powers is not a secret agent (4)"
1 specs in 0.115s.
>> 1 failures
Warning: Task "jasmine:src" failed. Use --force to continue.
Aborted due to warnings."Austin Powers is not a secret agent (4)" is a custom failure message. Also, the failed expect was the fourth one (4) according to the order mentioned in spec.
Test spies are objects that replace the actual functions with the code to record information about the function’s usage through the systems being tested. Spies are useful when determining a function’s success is not easily accomplished by inspecting its return value or changes to the state of objects with witch it interacts.
Consider the following example of login functionality. A showAuthorizedSection() function will be invoked within login function after the user entered the correct user name and password. We need to test that the invocation of the showAuthorizedSection() is happening in this sequence.
login function.var ssc = {};
(function() {
'use strict';
ssc.showAuthorizedSection = function() {
console.log("showAuthorizedSection");
};
ssc.login = function(usernameInput, passwordInput) {
// username and password check logic is omitted
this.showAuthorizedSection();
};
})();And here is how we can test it using Jasmine’s spies.
describe("login module", function() {
it("showAuthorizedSection has been called", function() {
spyOn(ssc, "showAuthorizedSection"); // (1)
ssc.login("admin", "1234"); // (2)
expect(ssc.showAuthorizedSection).toHaveBeenCalled(); // (3)
});
});-
The
spyOnfunction will replaceshowAuthorizedSection()function with corresponded spy. -
The
showAuthorizedSection()function will be invoked withinlogin()function in case of successful login. -
Assertion
toHaveBeenCalled()would be not possible without spy.
The previous section was about executing your test and specification in a headless mode using Grunt and PhantomJS, which is very useful for running tests in the CI environments. While PhantomJS uses WebKit rendering engine, there are browsers that don’t use WebKit. It’s obvious that running tests manually in each browser is tedious and not productive.
To automate testing in all Web browsers, you can use Testem Runner. Testem executes your tests, analyzes its output and prints result on the console. In this section you’ll learn how to install and configure Testem to run Jasmine tests.
Testem runner will just pick any of JavaScript in your project directory. If testem can identify any tests among that .js files it will run it. But Testem tasks can be customized using a configuration file.
You can configure Testem to specify which files should be included in testing. Testem starts with trying to find the configuration file testem.json in the project directory. A sample testem.json is shown in following listing.
{
"framework": "jasmine", // (1)
"src_files": [ // (2)
"ext/ext-all.js",
"test.js"
]
}-
The
frameworkdirective is used to specify the test framework. Testem supports QUnit, Jasmine and many more frameworks. You can find full list of supported frameworks on testem github page. -
The list of test and production code source files.
Testem supports two running modes: test-driven development mode (tdd-mode) and continuous integration (ci-mode). (For more about continuous integration, see the note on CI. In tdd-mode, testem starts the development server.
In tdd-mode, Testem doesn’t spawn any browser automatically. On the contrary, you’d need to open this url in the browser you want run test against to connect it to Testem server. From this point on, Testem will execute tests in all connected browsers. On the next screeshot you can see we added different browsers including mobile version of Safari (running on iOS simulator).
Because the Testem server itself is an HTTP server, you can connect remote browsers to it as well. For example, the following screenshot shows Internet Explorer 10 running under Windows 7 virtual machine connected to the testem server.
Running the tests with testem runner can be combined with previously introduced Grunt tool. The next screenshot shows two tests in parallel: testem runs tests on the real browsers and and grunt runs tests on the headless PhantomJS.
grunt watch side-by-sideTestem supports live reloading mode. This means that Testem will watch file system for changes and will execute tests in all connected browsers automatically. You can force to test run by switching to console and hitting the Enter key.
In CI mode testem will examine system for all of the available browsers in your system and will execute tests on it. You can get list of the browsers that testem can use to run tests with the testem launchers command. Here is sample output after running this command.
# testem launchers Have 5 launchers available; auto-launch info displayed on the right. Launcher Type CI Dev ------------ ------------ -- --- Chrome browser ✔ Firefox browser ✔ Safari browser ✔ Opera browser ✔ PhantomJS browser ✔
Now you can run our test simultaneously in all browsers installed in your computer - Google Chrome, Safari, Firefox, Opera and PhantomJS - with one command:
testem ci
testem ci command# Launching Chrome # (1)
#
# Launching Firefox # (2)
# ....
TAP version 13
ok 1 - Firefox Basic Assumptions: Ext namespace should be available loaded.
ok 2 - Firefox Basic Assumptions: ExtJS 4.2 should be loaded.
ok 3 - Firefox Basic Assumptions: SSC code should be loaded.
ok 4 - Firefox Basic Assumptions: something truthy.
# Launching Safari # (3)
#
# Launching Opera # (4)
# ....
ok 5 - Opera Basic Assumptions: Ext namespace should be available loaded.
ok 6 - Opera Basic Assumptions: ExtJS 4.2 should be loaded.
ok 7 - Opera Basic Assumptions: SSC code should be loaded.
ok 8 - Opera Basic Assumptions: something truthy.
# Launching PhantomJS # (5)
#
1..8
# tests 8
# pass 8
# ok
....-
The tests are run on Chrome…
-
… Firefox
-
… Safari
-
… Opera
-
… and on headless WebKit - PhantomJS
Testem uses TAP format to report test results.
As we discussed in Chapter 1, Document Object Model is a standard browser API that allows a developer to access and manipulate page elements. Pretty often your JavaScript code needs to access and manipulate the HTML page elements in some way. Testing DOM is the crucial part of testing your client side JavaScript. By design, the DOM standard defines a browser-agnostic API. But in the real world, if you want to make sure that your code works in the particular browser you need to run the test inside this browser.
Earlier in this chapter we’ve introduced the Jasmine method beforeEach(), which is the right place for setting all required DOM elements and making them available in the specifications.
describe("spec", function() {
var usernameInput;
beforeEach(function() { // (1)
usernameInput = $(document.createElement("input")).attr({ // (2)
type: 'text',
id: 'username',
name: 'username'
})[0];
});
});-
Inside the
beforeEach()method we’re using the API to manipulate the DOM programatically. Also, if you’re using an HTML test runner you can add the fixture using HTML tags. But we don’t recommend this approach because pretty soon you will find that the test runner will become unmaintainable and clogged with tons of fixture HTML code. -
Create an
<input>element using jQuery APIs, which will turn into the following HTML:
<input type="text" id="password" name="password" placeholder="password" autocomplete="off">The jQuery selectors API is more convenient for working with DOM than a standard JavaScript DOM API. But in the future examples we will use the jasmine-fixture library for easier setup of the DOM fixture. Jasmine-fixture uses similar to jQuery selectors syntax for injecting HTML fixtures. With this library you will significantly decrease the amount of repetitive code while creating the fixtures.
Let’s see how the example from previous code snippet looks like with the jasmine-fixture library.
describe("spec", function() {
var usernameInput;
beforeEach(function() {
usernameInput = affix('input[id="username"][type="text"][name="username]')[0]; // (1)
});
it("should not allow login with empty username and password and return code equals 0", function() {
var result = ssc.login(usernameInput, passwordInput); // (2)
expect(result).toBe(0);
});
});-
Using the
affix()function provided by the jasmine-fixture library and expressiveness of CSS selectors we can easily setup required DOM elements. More examples of possible selectors could be found at the documentation page of jasmine-fixture. -
Now when all requirements for our production code (
login()function) are satisfied we can run it in the context of a test and assert the results.
As you can see, testing of the DOM manipulation code is much like any other type of unit testing. You need to prepare a fixture (a.k.a. the testing context), run the production code and assert the results.
We assume that you’ve read the materials from Chapter 6, and in this section you’ll apply your newly acquired Ext JS skills. As a reminder, Ext JS framework encourages using MVC architecture. The separation of responsibilities between Views, Models and Controllers makes an ExtJS application a perfect candidate for unit testing. In this section you’ll learn how to test the Ext JS version of the Save The Child application from Chapter 6.
Let’s create a skeleton application that can provide for our classes under the test familiar environment.
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title id="page-title">ExtJS Jasmine Tester</title>
<link rel="stylesheet" type="text/css" href="test/lib/jasmine-1.3.1/jasmine.css"/>
<script type="text/javascript" src=ext/ext-all.js></script> // (1)
<script type="text/javascript" src="test/lib/jasmine-1.3.1/jasmine.js"></script> //(2)
<script type="text/javascript" src="test/lib/jasmine-1.3.1/jasmine-html.js"></script>
<script type="text/javascript" src="test.js"></script> // (3)
</head>
<body>
</body>
</html>-
Adding the Jasmine framework dependencies.
-
Adding ExtJS framework dependencies.
-
This is our skeleton ExtJS application that will setup "friendly" environment for components under the test. You can see content of test.js in the following listing.
Ext.Loader.setConfig({
disableCaching: false,
enabled: true,
paths: {
Test: 'test', // (1)
SSC: 'app' // (2)
}
});
var application = null;
Ext.onReady(function() {
application = Ext.create('Ext.app.Application', {
name: 'SSC', // (3)
requires: [
'Test.spec.AllSpecs' // (4)
],
controllers: [
'Donate' // (5)
],
launch: function() {
Ext.create('Test.spec.AllSpecs');
jasmine.getEnv().addReporter(new jasmine.TrivialReporter()); // (6)
jasmine.getEnv().execute();
}
});
});-
Ext JS loader needs to know the location of the testing classes…
-
… and about location of production code.
-
Create a skeleton application in the namespace of the production code to provide the execution environment.
-
The
AllSpecclass will be requesting loading of the rest of the specs. We will show code ofAllSpecclass in next listing -
The skeleton application will test the controllers from the production application code
-
This code initializes Jasmine environment and reporters.
Ext.define('Test.spec.AllSpecs', {
requires: [ // (1)
'Test.spec.BasicAssumptions'
/*, 'Test.spec.ModelAssumptions' */
]
});-
The
requiresproperty includes an array of Jasmine suites. All further tests will be added to this array. Ext JS framework will be responsible for loading and instantiation all test classes.
Here is how our typical test suite will be look like.
Ext.define('Test.spec.BasicAssumptions', {}, function() { // (1)
describe("Basic Assumptions: ", function() { // (2)
it("Ext namespace should be available loaded", function() {
expect(Ext).toBeDefined();
});
it("SSC code should be loaded", function() {
expect(SSC).toBeDefined();
});
});
});-
Wrap the Jasmine suite into an Ext JS class.
-
The rest of the code is very similar to the Jasmine code sample shown earlier in this chapter.
After setting up the testing harness for Save The Child application we will suggest testing strategy for ExtJS applications. Let’s start with testing the models and controllers followed by testing the views.
SaveSickChild.org home page displays the information about fund raising campaigns using chart and table views backed by collection of Campaign models. A Campaign model should have three properties: title, description, and location. The title property of the model should have a default value - Default Campaign Title. The location property of the model is required field.
In a spirit of TDD, let’s write the specification what will meet the requirements described above.
link:include/CampaignModelAssumptions.js[role=include]-
By default,
Ext.data.Modelcaches every model created by the application in a global in-memory array. We need to clean up the ExtJS model cache after each test run. -
Instantiate the
Campaignmodel class to check that it exists. -
Next we need to check if model has all required properties.
-
The property
titlehas a default value. -
Validation will fail on the empty
locationproperty.
link:include/Campaign.js[role=include]Controllers in ExtJS are classes like any others and should be tested the same way. In the next example, let’s test Donate Now functionality. When the user clicks Donate Now button of the Donate panel controller’s code should validate the user input and submit the data to the server. Since we are just testing controller’s behavior, we’re not going to submit the actual data. We’ll use Jasmine spies instead.
Ext.define("Test.spec.DonateControllerSpec", {}, function () {
describe("Donate controller", function () {
beforeEach(function () {
// controller's setup code is omitted
});
it('should exists', function () { // (1)
var controller = Ext.create('SSC.controller.Donate');
expect(controller.$className).toEqual('SSC.controller.Donate');
});
describe('donateNow button', function () {
it('calls donate on DonorInfo if form is valid', function () {
var donorInfo = Ext.create('SSC.model.DonorInfo', {});
var donateForm = Ext.create('SSC.view.DonateForm', {});
var controller = Ext.create('SSC.controller.Donate');
spyOn(donorInfo, 'donate'); // (2)
spyOn(controller, 'getDonatePanel').andCallFake(function () { // (3)
donateForm.down = function () {
return {
isValid: function () {
return true;
},
getValues: function () {
return {};
}
};
};
return donateForm;
});
spyOn(controller, 'newDonorInfo').andCallFake(function () { //(4)
return donorInfo;
});
controller.submitDonateForm();
expect(donorInfo.donate).toHaveBeenCalled(); // (5)
});
});
});
});-
First, you need to test if controller’s class is available and can be instantiated.
-
With the help of Jasmine’s
spyOn()function substitute theDonorInfomodel’sdonate()function. -
We’re not interested in the view’s interaction - only the contract should be tested. At this point, some methods can be substituted with the fake implementation to let the test pass. In this case, the specification tests the situation when form’s valid.
-
Next you need to inject emulated controller dependencies. The function
donate()was replaced by the spy. -
Finally, you can assert if the function was called by the controller.
The function under the test looks as follows:
Ext.define('SSC.controller.Donate', {
extend: 'Ext.app.Controller',
refs: [{
ref: 'donatePanel',
selector: '[cls=donate-panel]'
}
],
init: function() {
'use strict';
this.control({
'button[action=donate]': {
click: this.submitDonateForm
}
});
},
newDonorInfo: function() { // (1)
return Ext.create('SSC.model.DonorInfo', {});
},
submitDonateForm: function() {
var form = this.getDonatePanel().down('form');
if (form.isValid()) { //(2)
var donorInfo = this.newDonorInfo();
Ext.iterate(form.getValues(), function(key, value) { //(3)
donorInfo.set(key, value);
}, this);
donorInfo.donate(); // (4)
}
}
});-
The factory method for creating a new instance of the
SSC.model.DonorInfoclass. -
If the form is valid, read data from the form fields…
-
… and populate properties of corresponding object.
-
DonorInfocan be submitted by calling thedonate()method.
UI Tests can be divided into two constituent parts: interaction tests and component tests.Interactions tests simulate real-world scenarios of application usage as if a user is using the application. It’s better to delegate the interaction tests to the functional testing tools like Selenium or CasperJS.
There is another UI testing tool worth to mention especially, in the context of testing ExtJS applications - Siesta. Siesta allows to perform testing of the DOM and simulate user interactions. Siesta written in JavaScript and uses Siesta for unit and UI testing. There are two editions of Siesta - lite and professional.
Component Tests isolate independent and reusable pieces of your application to verify their display, behavior and contract with other components (see testing of the controllers). Let’s see how we can do that. Consider following example
Ext.define('Test.spec.ViewsAssumptions', {}, function () {
function prepareDOM(obj) { //(1)
Ext.DomHelper.append(Ext.getBody(), obj);
}
describe('DonateForm ', function () {
var donateForm = null; //(2)
beforeEach(function () {
prepareDOM({tag: 'div', id: 'test-donate'}); // (3)
donateForm = Ext.create('SSC.view.DonateForm', { //(4)
renderTo: 'test-donate'
});
});
afterEach(function () {
donateForm.destroy(); // (5)
donateForm = null;
});
it('should have donateform xtype', function () {
expect(donateForm.isXType('donateform')).toEqual(true); // (6)
});
});
});-
A helper function for fixture DOM elements creation.
-
A reusable scoped variable.
-
Create fixture
testdiv. -
Create a fresh form for every test to avoid test pollution.
-
Destroy the form after every test so we don’t pollute the environment.
-
In this test, you need to make sure that the
DonateFormcomponent hasdonateformxtype.
In this section we will setup WebStorm to use the described above tools inside this IDE. We will show how to integrate Grunt tool with WebStrom to run grunt tasks from there.
Let’s start with the Grunt setup. Currently, there is no native support of the Grunt tool in WebStorm IDE. Since Grunt is a command line tool, you can use a general launching feature of the WebStorm IDE and configure it as an External Tool. Open the WebStorm preferences and navigate to External Tools section to get access to the external tools configuration as in External Tools configuration window in WebStorm.
Click the + button to create new External Tool configuration.
-
You need to specify the full path to the application executable.
-
Some tools require command line parameters. In this example, we explicitly specify the task runner configuration file (with the
--gruntiflecommand line option) and the task to be executed. -
Also you need to specify the Working Directory to run the Grunt tool. In our case, grunt configuration file is located in the root of our project. WebStorm allows to use macros to avoid hard-coded paths. Most likely, you don’t want to setup external tools for each new project, just create a universal setup. In our example we use the
$ProjectFileDir$macros will be resolved as current WebStorm project folder root. -
WebStorm allows you to organize related tasks into logical groups.
-
You can configure how to access the external tool launcher.
When all of the above steps are complete you can find the launcher under Tools menu as well under Main menu Editor menu, Project views and etc.
Unit tests are really important as a mean to get a quick feedback from your code. You can work more efficient if you manage to minimize context switching during your coding flow. Also, you don’t want to waste time digging through the menu items of your IDE, so assigning a keyboard shortcut for launching external tool is a good idea.
Let’s assign a keyboard shortcut for our newly configured external tool launcher. Go to the Keymap section in WebStorm Preferences. Use the filter to find our created launcher jasmine: grunt test. Specify either the Keyboard of the Mouse shortcut by double clicking on the appropriate list item.
By pressing a combination of keys specified in the previous screen, you will be able to launch the grunt with Jasmine tests with one click of a button(s). WebStorm will redirect all the output from the grunt tool into its Run window.
Testing is one of the most important processes of software development. Well organized testing helps keeping the code in a good and working state. It’s especially important in interpreted languages like JavaSctipt where there is no compiler to provide a helping hand to find lots of errors on very early stages.
In this situation, static code analysis tools, like JSHint (discussed in «Selected Productivity Tools for Enterprise Developers» chapter), could become very handy in helping with identifying typos and enforcing best practices accepted by the JavaScript community.
In enterprise projects developed with compiled languages people often debate if test-driven development is really beneficial. With JavaScript it’s non-debatable unless you have unlimited time and budget and are ready to live with unmaintainable JavaScript.
The enterprises that have adopted test-driven development (as well as behavior-driven development) routines make the application development process safer by including test scripts in the continuous integration build process.
Automating unit tests reduces the number of bugs and decreases the amount of time developers need to spend manually testing their code. If automatically launched test scripts (unit, integration, functional, and load testing) don’t reveal any issues, you can rest assured that the latest code changes did not break the application logic, and that the application performs according to SLA.