- Test runners
- Unit testing frameworks
- Integration testing frameworks
- Assertion frameworks
- Mocking frameworks
If you’ve used some of these, you might have a preference based on what feels right for you. If you haven't, you might use a different criteria: popularity.
If you don’t have a preference, picking the most popular test framework is useful for a few reasons:
- You are crowdsourcing the research. If the most people are using a particular framework, there’s often a reason for that. It might not be the best framework, or the up-and-coming framework, but it’s probably mature and reasonably solid.
- If you have an issue, you probably won’t be the first person to have that issue, and finding answers to your problems is often easier.
- When hiring or outsourcing, it’s a lot easier to find someone with experience with the popular frameworks than the niche frameworks.
Gauging PopularityOf course, if you’re trying to assess the popularity of a tool, there’s a bunch of different ways you could do that, including:
QUnitQUnit is a unit testing framework that was originally part of JQuery.
- NPM: ~20k downloads in the last month. Often over 500 downloads per day.
- Job Trends: QUnit was initially a strong contender against Jasmine until early 2013, after which it has stayed steady while Jasmine has quadrupled. In early 2014, Mocha also surpassed it and has now doubled.
Basically, QUnit has been around for a while, it’s mature, but it’s clearly not at the same popularity level as Jasmine and Mocha. It’s probably not a terrible choice, but those people looking for the popular framework should focus their energy on either Jasmine or Mocha instead.
JasmineJasmine is a behaviour-driven test framework, intended to be a fairly complete system on its own – it comes with assertions, spies, mocking.
- NPM: Almost ~300k downloads in the last month. Was hitting ~1k/day in 2014, regularly over 5k/day in Q1 2015 and now often over 10k/day.
- Stack Overflow: Jasmine is the leader in tag trends and total questions, but the data is close and varies wildly.
- Google: Jasmine got the early lead and has held it. Mocha is making some headway.
MochaTest framework designed with an extension model in mind.
- NPM: Mocha seems to dominate NPM downloads, with 2M downloads in the last month, and many days over 50k downloads/day this year.
- Job Trends: Mocha is a relative newcomer, but has already surpassed QUnit and is making headway on Jasmine. Still, Jasmine is featured in twice as many job postings as Mocha, so it’s still the underdog.
- Stack Overflow: Half of Jasmine’s total questions, but pretty competitive on tag trends.
- Google: Mocha got the latest start on the search trends, but has already surpassed QUnit. It’s not clear if it’s making headway on Jasmine on search trends.
- Stackshare: Mocha handily beats Jasmine on Stackshare. I’m not sure if people are more happy to share that they’re using Mocha (e.g. aspirational) or if the kinds of people on Stackshare are more likely to use Mocha (e.g. selection bias).
KarmaKarma is a test runner, popularized by AngularJS. It is test framework agnostic, and works with QUnit, Jasmine and Mocha.
- NPM: Almost 1M downloads in the last month and regularly over 30k dowloads/day this year.
- Google: Karma’s pretty recent on Google search trends, but it’s already competitive with Mocha in terms of search.
ChaiChai is an assertion framework, but one that has an extension model of its own, allowing yet another layer of library to layered on top of it.
- NPM: ~850k downloads in the last month and regularly over 20k downloads per day. Chai is about twice as popular as Should, but they’re in the same league.
- Job Trends: Chai is the only assertion framework to have meaningful results on job trends. Of course, doesn’t help that its two strongest competitors, expect.js and should.js have more generic names that make searching a little harder.
- Stack Overflow: Doesn’t seem to place on tag trends, but well ahead of should.js on total questions.
- Google: Up alongside QUnit, but below Mocha and Jasmine.
Should.jsShould is a BDD-style assertion framework for node and browsers.
- NPM: Almost 400k downloads int he last month, and regularly over 10k/day downloads this year.
- Stack Overflow: Well behind Chai on total questions.
- Google: Hard to get clean results, but “should.js” is clearly below Chai.
OthersThere are, of course, countless others. Some of them are specific to a particular platform, but many are simply alternatives to those listed above, or another tool related to testing but not fitting within the broad categorization above.
I’ve tried to cover the most popular set, but there are lots of others on the long tail of popularity, such as:
- Expect is another BDD-style assertion framework, built by Automattic, for node.js and browser testing. At about 200 downloads/day and <10k downloads in the last month, it’s clearly far less popular than Should and Chai.
- Wallaby is another test runner.
- NodeUnit is a test framework built on node.js’s assertions.
- JSTestDriver is another test runner.
- Dalek is more of an end-to-end framework, with a runner, assertions, browser support and more.
- Atomus is a test framework for client-side tests in a node environment by simulating a browser.
- Sinon adds spies, stubs and mocks, and is often used with Chai.
- Protractor is an end-to-end testing tool for AngularJS.
- Although code coverage tools aren’t specific to testing, they are often used in conjunction with testing. Some of the candidates here are:
- Nightwatch is an end-to-end testing framework that works with Selenium and node.js.
- Jest is layered on top of Jasmine with the goals of making Jasmine that much easier to use (“painless”).
What Should I Use?That’s ultimately your decision, but if popularity is your primary criterion, then Karma with either Jasmine or Mocha & Chai are clearly the leaders.
If you want to talk about this or something else, call me on Clarity, or get in touch.