Wednesday, August 12, 2015

Mocking HTTP Servers

I am doing some HTTP-based integration work for a client on a GWT-based project (which is why I wanted to know which HttpClient version to use). I'm still waiting on access to the third-party system, but I've already written and tested the code to generate the XML request and parse the XML response, so I wanted to take a step farther and write the code that would connect that with HTTP:

  • Generate XML using the XML writer/generator
  • Send the XML over HTTP
  • Wait for the HTTP Response, check it (status code, etc)
  • Parse the HTTP response's body using the XML parser/reader

But if I were going to do that, I want to test it, and testing it requires something to listen for HTTP requests (or to mock out the HTTP client, but this is already a pretty thin layer). I don't have access to the real service yet, so I can't write an integration test and, to be honest, I tend to try not to rely on integration tests. So basically I want something that can pretend to be an HTTP Server of some kind but that has an API that makes writing test code convenient. I went looking for candidates to easily test an HTTP-based client in Java.

When I'm picking tools in other platforms, I often consider the popularity of the dependency as well as other signs of its maturity. In Java, it's hard to get good numbers for such things. First of all, while there are some dependency management systems for Java, they're not universal or nearly-universal like they are on other platforms. Maven's central repository is probably the closest thing to that, but even then, there are lots of projects that download their dependencies directly. And, sadly, Maven doesn't publish stats about their dependencies, stats you could use to gauge how many other people are using and trusting a particular library vs. another.

However, after looking at some of the alternatives, I decided to start with mocky.io. That worked pretty well, but I don't really love using an external server for a simple test. That test wouldn't run offline, wouldn't run if the internet was down, or if mocky.io was having problems. It's basically too much of an integration test, and not even an integration test against the real service. Still, it worked well, and it's a reasonable place to start if you just want to fire up a simple test without adding new dependencies to your project.

I looked at alternatives to replace it. WireMock has a reputation for being very capable and having a good API, but perhaps not kept as up-to-date as one might like and a little heavy-weight. I also looked at MockServer, which looks very capable but also looks pretty heavy. My requirements for now are super-simple, so I ended up using okhttp's MockWebServer, one of Square's open-source projects. It's very simple, reasonably light, very fast, and has enough capability for my simple needs. I like it.

There are lots of alternatives, from something custom (Netty, RxNetty, Spray) to other competitive libraries (jailer,mock-http-server) and services (Sandbox), but for the moment, MockWebServer does the job for me.

Monday, August 10, 2015

GWT / HttpClient Version Compatibility

If you're using GWT and Apache HttpClient in a single project, it can be useful to know which version of HttpClient ships with which version of GWT, because GWT includes HttpClient directly in its jar, and conflicting versions can be a bit of a pain.

I can't find that information published anywhere else, so I'm publishing it here. Basically, look in the dev/build.xml file for the tagged version of GWT that you care about. Since I've already done that, here's the result for several of the most recent releases:

GWT VersionHttpClient Version
2.74.3.1
2.6.x4.3.1
2.5.x4.1.2
2.44.0.1

I hope this will be useful for someone else.

Friday, June 19, 2015

Web Assembly is Backed by the Right People

There have been a few attempts over the years to take a more revolutionary approach to pushing the web forward. Google's Dart, Microsoft's TypeScript are some examples of these. While these have been interesting to skim, I have never followed them too closely because they lacked the kind of support they would need to succeed. JavaScript alternatives can compile to JavaScript, but without being built into the browser, they were never going to replace JavaScript.

Yesterday, news of the Web Assembly initiative broke:
It's far too early to decide whether or not this will work out, there's very little detail to go on at this point. However, I was pleased to see that the group of companies putting some support behind it seems to be the right group. If Apple, Mozilla, Microsoft and Google all put their weight behind an initiative to improve the web, there's a much better chance of it succeeding than if one, or none of them does.

Now, "weight" is a subjective term, and it wouldn't be hard to imagine one or all of these companies not putting enough force behind this to make it a success. But if they really do all commit significant resources to this kind of effort, there's a real chance that this could be a moment in the history of the web that we look back on as a turning point.

It's still too early to have much of an opinion, but I'm interested to see where this goes.

Monday, May 25, 2015

Javascript Testing Frameworks

On a recent Clarity call, one of the topics we discussed was JavaScript Testing Frameworks.

It’s a complex topic, more complex than it is in many other platforms, because JavaScript testing frameworks are pretty granular – there are lots of pieces and they can be used together or independently.

There are:
  • Test runners
  • Unit testing frameworks
  • Integration testing frameworks
  • Assertion frameworks
  • Mocking frameworks
Some tools are very specifically just one of these. Other tools cover more than one category or blur the lines between several of these.

If you’ve used some of these, you might have a preference based on what feels right for you. If you haven't, you might use a different criteria: popularity.

If you don’t have a preference, picking the most popular test framework is useful for a few reasons:
  • You are crowdsourcing the research. If the most people are using a particular framework, there’s often a reason for that. It might not be the best framework, or the up-and-coming framework, but it’s probably mature and reasonably solid.
  • If you have an issue, you probably won’t be the first person to have that issue, and finding answers to your problems is often easier.
  • When hiring or outsourcing, it’s a lot easier to find someone with experience with the popular frameworks than the niche frameworks.
Of course, using the popular frameworks isn't likely to give you a competitive advantage, but it's up to you to decide when it's worth seeking that kind of advantage. I'm not sure that Javascript testing frameworks is the first place I'd seek it.

Gauging Popularity

Of course, if you’re trying to assess the popularity of a tool, there’s a bunch of different ways you could do that, including:

Candidates

What are some of the primary candidates for JavaScript and testing?

QUnit

QUnit is a unit testing framework that was originally part of JQuery.

Popularity:
  • NPM: ~20k downloads in the last month. Often over 500 downloads per day.
  • Job Trends: QUnit was initially a strong contender against Jasmine until early 2013, after which it has stayed steady while Jasmine has quadrupled. In early 2014, Mocha also surpassed it and has now doubled.
  • Stack Overflow: a distant third in total questions, and rarely top of the trending tags for Javascript test frameworks.
    Basically, QUnit has been around for a while, it’s mature, but it’s clearly not at the same popularity level as Jasmine and Mocha. It’s probably not a terrible choice, but those people looking for the popular framework should focus their energy on either Jasmine or Mocha instead.

Jasmine

Jasmine is a behaviour-driven test framework, intended to be a fairly complete system on its own – it comes with assertions, spies, mocking.

Popularity:
  • NPM: Almost ~300k downloads in the last month. Was hitting ~1k/day in 2014, regularly over 5k/day in Q1 2015 and now often over 10k/day.
  • Job Trends: Jasmine is the leading JavaScript testing framework, although Mocha has been making steady progress, and is probably the strongest alternative.
  • Stack Overflow: Jasmine is the leader in tag trends and total questions, but the data is close and varies wildly.
  • Google: Jasmine got the early lead and has held it. Mocha is making some headway.

Mocha

Test framework designed with an extension model in mind.

Popularity:
  • NPM: Mocha seems to dominate NPM downloads, with 2M downloads in the last month, and many days over 50k downloads/day this year.
  • Job Trends: Mocha is a relative newcomer, but has already surpassed QUnit and is making headway on Jasmine. Still, Jasmine is featured in twice as many job postings as Mocha, so it’s still the underdog.
  • Stack Overflow: Half of Jasmine’s total questions, but pretty competitive on tag trends.
  • Google: Mocha got the latest start on the search trends, but has already surpassed QUnit. It’s not clear if it’s making headway on Jasmine on search trends.
  • Stackshare: Mocha handily beats Jasmine on Stackshare. I’m not sure if people are more happy to share that they’re using Mocha (e.g. aspirational) or if the kinds of people on Stackshare are more likely to use Mocha (e.g. selection bias).
Mocha is clearly a credible alternative to Jasmine, and seems better suited to people who want to mix and match pieces of the framework rather than relying on a single complete solution. I feel like Mocha is ‘in fashion’ or ‘trending’ right now now. Still, I don’t think you’ll go that far wrong with either Mocha or Jasmine.

Karma

Karma is a test runner, popularized by AngularJS. It is test framework agnostic, and works with QUnit, Jasmine and Mocha.

Popularity:
  • NPM: Almost 1M downloads in the last month and regularly over 30k dowloads/day this year.
  • Job Trends: Karma is the only JavaScript test runner with meaningful penetration on job postings that I could find, although it’s certainly less common than any of the test frameworks.
  • Stack Overflow: Vastly ahead of all the other JavaScript test runners on total questions.
  • Google: Karma’s pretty recent on Google search trends, but it’s already competitive with Mocha in terms of search.

Chai

Chai is an assertion framework, but one that has an extension model of its own, allowing yet another layer of library to layered on top of it.

Popularity:
  • NPM: ~850k downloads in the last month and regularly over 20k downloads per day. Chai is about twice as popular as Should, but they’re in the same league.
  • Job Trends: Chai is the only assertion framework to have meaningful results on job trends. Of course, doesn’t help that its two strongest competitors, expect.js and should.js have more generic names that make searching a little harder.
  • Stack Overflow: Doesn’t seem to place on tag trends, but well ahead of should.js on total questions.
  • Google: Up alongside QUnit, but below Mocha and Jasmine.

Should.js

Should is a BDD-style assertion framework for node and browsers.

Popularity:
  • NPM: Almost 400k downloads int he last month, and regularly over 10k/day downloads this year.
  • Job Trends: Although it’s harder to search for “javascript” and “should” and get useful results, neither “shoud.js” nor “shouldjs” seem to show up on significant numbers of job posts.
  • Stack Overflow: Well behind Chai on total questions.
  • Google: Hard to get clean results, but “should.js” is clearly below Chai.

Others

There are, of course, countless others. Some of them are specific to a particular platform, but many are simply alternatives to those listed above, or another tool related to testing but not fitting within the broad categorization above.

I’ve tried to cover the most popular set, but there are lots of others on the long tail of popularity, such as:
  • Expect is another BDD-style assertion framework, built by Automattic, for node.js and browser testing. At about 200 downloads/day and <10k downloads in the last month, it’s clearly far less popular than Should and Chai.
  • Wallaby is another test runner.
  • NodeUnit is a test framework built on node.js’s assertions.
  • PhantomJS is a headless WebKit with scriptable JavaScript API which can be used for testing.
  • JSTestDriver is another test runner.
  • Cucumber.js is a behaviour-driven development (BDD) framework for JavaScript.
  • Dalek is more of an end-to-end framework, with a runner, assertions, browser support and more.
  • Atomus is a test framework for client-side tests in a node environment by simulating a browser.
  • Testem is another JavaScript test runner. At 200k NPM installs in the last month, it’s probably the most credible alternative to Karma.
  • Sinon adds spies, stubs and mocks, and is often used with Chai.
  • Protractor is an end-to-end testing tool for AngularJS.
  • Although code coverage tools aren’t specific to testing, they are often used in conjunction with testing. Some of the candidates here are:
  • Nightwatch is an end-to-end testing framework that works with Selenium and node.js.
  • Jest is layered on top of Jasmine with the goals of making Jasmine that much easier to use (“painless”).
  • Chutzpah is another JavaScript test runner that seems to be built with the Microsoft stack at least partially in mind.

What Should I Use?

That’s ultimately your decision, but if popularity is your primary criterion, then Karma with either Jasmine or Mocha & Chai are clearly the leaders.

If you want to talk about this or something else, call me on Clarity, or get in touch.

Tuesday, April 21, 2015

Wrapping Up a CITS eApp Notification Feed

I'm in the late stages of wrapping up a project to add a CITS eApp Notification feed as a integration between an insurance client of mine and a vendor's system (bluesun's WealthServ), and I thought it might be helpful to look back over the effort and share things that I learned anew or, at least, things I received fresh reminders of along the way.

If you haven't encountered some of these terms elsewhere, let me explain:

  • CITS is the Canadian Insurance Transaction Standardization effort by CLIEDIS.
  • CLIEDIS is the Canadian Life Insurance EDI Standards organization.
  • eApp Notification feed is a feed to inform vendors of an eApplication submission. Basically, it carries core information about a new application from one organization to another.

For instance, an insurance company might want to send information about new policies in the application process to, say, WealthServ or VirtGate so that brokers could login and see their policy, and get updates and commission information.

I've been taking notes in an outline to write up, and the outline is now big enough that I'm sure I'm going to need more than one post. So I'm starting here with a brief description of the project and some of the terms, and then I'll follow that with a series of posts covering many of these topics:

  • Integration Projects and Models
  • Generic Data Models
  • Monitoring and Logging
  • Security

That's not an exhaustive list, but it's a start. More to come.

Tuesday, August 12, 2014

Git Tower 2 Recommendation: Cautious Buy (Update: Buy)

I've been using Git Tower for a couple of years now, and I'm happy to recommend v1.x as a solid GUI Git client for OS X. There are a few things I'd like to see improved, but the basics are quite good, and it's been relatively bug-free for me (although after doing a Twitter search, I can see some people had some problems with v1.x that I didn't).

Since Git Tower v2.0 was released, I've installed a trial and I've been using it on and off in parallel with Git Tower v1.x. In general, I like it. I think it's a fine evolution of Git Tower, I think the UI changes are mostly improvements, and I'm looking forward to using it full-time.

Having said that, I don't think it's ready for prime-time use, at least not for me. I've reported a few bugs already while using the trial, and I'm not the only one. And if you read over the mentions that @gittower has on Twitter, I think that's the general sentiment: looks good, but needs a bit more attention.

There has been a few point releases already (v2.0.3 at the time of this writing) and I expect there will be more. Basically, I'd say it looks like a promising upgrade, but if you're in no rush, it wouldn't hurt to wait a little longer. If you're curious, then download the trial in the meantime. If you really want to use Tower 2, you probably can, but accept there might be a few issues along the way.

Update: I have been using Git Tower 2 for months now and almost all the issues have been worked out. It still, isn't perfect, but I don't think the remaining issues warrant much caution. I would say "buy".

Wednesday, June 11, 2014

TweetDeck XSS Vulnerability in Desktop Application

TweetDeck's recent XSS vulnerability highlights one of the downsides to using web technologies to build a desktop app.

Of course, no app is immune to vulnerabilities, and using desktop application platforms certainly doesn't prevent all vulnerabilities. But TweetDeck's vulnerability exists in part because the content it displays is delivered in the same form as its user interface.

A desktop application that failed to properly control web content that it was displaying would still expose the web container in which the content was running, but that wouldn't make it vulnerable to manipulating the application controls. A badly secured web view to display a tweet within in a native application with native controls for displaying tweets would still make it hard to, say, trigger the application to retweet the content automatically.