Showing posts with label java. Show all posts
Showing posts with label java. Show all posts

Wednesday, August 12, 2015

Mocking HTTP Servers

I am doing some HTTP-based integration work for a client on a GWT-based project (which is why I wanted to know which HttpClient version to use). I'm still waiting on access to the third-party system, but I've already written and tested the code to generate the XML request and parse the XML response, so I wanted to take a step farther and write the code that would connect that with HTTP:

  • Generate XML using the XML writer/generator
  • Send the XML over HTTP
  • Wait for the HTTP Response, check it (status code, etc)
  • Parse the HTTP response's body using the XML parser/reader

But if I were going to do that, I want to test it, and testing it requires something to listen for HTTP requests (or to mock out the HTTP client, but this is already a pretty thin layer). I don't have access to the real service yet, so I can't write an integration test and, to be honest, I tend to try not to rely on integration tests. So basically I want something that can pretend to be an HTTP Server of some kind but that has an API that makes writing test code convenient. I went looking for candidates to easily test an HTTP-based client in Java.

When I'm picking tools in other platforms, I often consider the popularity of the dependency as well as other signs of its maturity. In Java, it's hard to get good numbers for such things. First of all, while there are some dependency management systems for Java, they're not universal or nearly-universal like they are on other platforms. Maven's central repository is probably the closest thing to that, but even then, there are lots of projects that download their dependencies directly. And, sadly, Maven doesn't publish stats about their dependencies, stats you could use to gauge how many other people are using and trusting a particular library vs. another.

However, after looking at some of the alternatives, I decided to start with mocky.io. That worked pretty well, but I don't really love using an external server for a simple test. That test wouldn't run offline, wouldn't run if the internet was down, or if mocky.io was having problems. It's basically too much of an integration test, and not even an integration test against the real service. Still, it worked well, and it's a reasonable place to start if you just want to fire up a simple test without adding new dependencies to your project.

I looked at alternatives to replace it. WireMock has a reputation for being very capable and having a good API, but perhaps not kept as up-to-date as one might like and a little heavy-weight. I also looked at MockServer, which looks very capable but also looks pretty heavy. My requirements for now are super-simple, so I ended up using okhttp's MockWebServer, one of Square's open-source projects. It's very simple, reasonably light, very fast, and has enough capability for my simple needs. I like it.

There are lots of alternatives, from something custom (Netty, RxNetty, Spray) to other competitive libraries (jailer,mock-http-server) and services (Sandbox), but for the moment, MockWebServer does the job for me.

Friday, January 17, 2014

Maven XML Verbosity is Amusing

Let's say you need to do something very slightly off the beaten path with Maven, like extract a single file from one of your dependencies so that it will be available to an XHR request in an offline web app as a static resource. No problem, just enter a quick 18 lines of XML.

But wait -- what if you're using Eclipse and M2E, and M2E decides that it doesn't know what to do about that dependency unpack you've just added? No problem, another 27 lines of XML oughta fix that.

Even though I know this to be true about Maven, every time I actually have to do it, it amuses me yet again.

Ah, Maven.

Monday, May 27, 2013

Separating MVEL from Moo

A feature request earlier in the year for Moo, my framework for mapping from objects to objects, was to support accessing private fields as the source for a translation as well as the destination.

This was a bit of a problem, because ever since MVEL was integrated with Moo, Moo has been using MVEL to access all source properties, and I wasn't able to find a way to access a private field using MVEL.

I'm also sensitive to the fact that MVEL is an additional dependency, and while it adds some flexibility to what Moo can accomplish, not everyone needs that flexibility and wants that dependency. The final nails in the coffin are a very low-traffic mailing list where questions often go unanswered and a few bugs that I've been bitten by that don't seem to be getting resolved. Essentially, it's not something I trust completely and as a result, I don't really want Moo to be as tightly coupled to it as it has been in the past.

Changing this, however, is a bunch of work. I'm not looking to remove MVEL altogether, just loosen the couplings so that people can choose to use or not use MVEL when they use Moo. This meant separating what was once a single project into two, giving them a multi-module build, adding an extension mechanism to moo-core that moo-mvel can use, and so forth.

That work is largely complete, but since it was so much work, I picked a bunch of other features that I've been meaning to enhance for some time and threw together a 2.0 release. That work is still underway, but I hope it will be done in the relatively near future.

This leads directly into another topic I've been itching to write up -- dependency relationships. But that's a topic for another post.

Tuesday, April 16, 2013

Hibernate 4 Still Maturing a Year Later?

In December 2011, the Hibernate team announced the release of Hibernate 4.0.Final. A year and a third later, it seems like Hibernate 4.x is still not as mature as I would have liked it to be on launch, let alone with all this extra time to add finesse. This makes me question the viability of the 4.0 line whether or not there are deeper problems with the project and team that are visible on the surface.

Project Background
A client of mine has a project that is now almost four years in development (over multiple releases, of course), and that started with Hibernate 3.3.x, which was current at the time.

During a number of those releases, the project team considered updates on various dependencies, including Hibernate, but hadn't found a compelling reason to upgrade. Since any upgrade introduces some change and risk, this project has continued to use 3.3.x.

A bug filed against that project recently showed that Hibernate's handling of character database columns (e.g. CHAR(10)) was somewhat flawed. This defect is fixed in Hibernate 3.6.8 and Hibernate 4.0.0.CR5, so it was time to evaluate Hibernate for an upgrade.

Evaluating Hibernate 4.x
Since Hibernate 4.x had been out for over a year, it seemed like it was probably worth serious consideration as the target version for the upgrade. I discarded 4.2.0.Final because it was relatively recent, and did a trial upgrade to 4.1.11.Final. The 4.1 line has been around since early February 2012, so it is also more than a year old.

There were immediately some issues, which I expected -- APIs have changed, deprecated methods removed, and so forth. This is pretty normal for an upgrade. But over the course of solving those problems, I got the strong impression that Hibernate 4.x still hasn't matured. After fighting with that for a while, I eventually decided that I would drop down to 3.6.10.Final, which is where I left it.

JDBC Connection and Work API
One of the first stumbling blocks was the removal of the deprecated method to get the JDBC connection underlying the Hibernate session. There are a few places where this project interacts directly with JDBC, because it's the best fit for that particular situation, and rather than having two mechanisms for accessing the database, we borrow Hibernate's JDBC connection to do a little bit of work.

Hibernate has allowed you to access the underlying JDBC connection for a long time, but in Hibernate 3.3.0 (I believe), this methods was deprecated and a new API where a connection is passed into a custom class implementing org.hibernate.jdbc.Work was introduced. This allows Hibernate to manage the JDBC connection more directly, which seems like a sensible idea.

In Hibernate 4.x, the deprecated Session.connection() method was removed. I'm in favour of removing deprecated methods, so this didn't bother me, and I set about modifying the project code to factor this in. It added some awkward indirection in our existing approaches, but I could see that a more involved refactoring might make that less awkward.

However, I also discovered that the Work API isn't covered in the Hibernate documentation (it's only even mentioned once that I can find), which still refers to Session.connection (e.g. Developer Guide, Reference Documentation), which has already been removed.

When Hibernate 4.x was first released, referring to methods that have been removed in the documentation would have just been an oversight. Over a year later, it seems clear that this isn't something that's just about to be fixed.

Bootstrapping Hibernate
For the longest time, bootstrapping Hibernate went a little like this:

  Configuration.configure().buildSessionFactory();

Hibernate 4.x involved a significant change the plumbing of Hibernate. The release notes mention both "Clean up of Session opening from SessionFactory" and "Introduction of ServiceRegistry". In the process, they deprecated buildSessionFactory() in favour of buildSessionFactory(ServiceRegistry). Wanting to better understand this change and learn the proper replacement for the above stanza, I went to the documentation.

Unfortunately, here's another case where the documentation is out-of-date, continuing to refer to now-deprecated initialization approach: Getting Started, Reference Manual. In looking at the source code comments for Configuration, I found references to Jira issues talking about deprecation (and that Configuration would be removed in Hibernate 5.0): HHH-6183, HHH-2578 and HHH-6586. Reading these and some related issues implies that the hibernate team:

Given that this change seems like a big part of what's included in 4.x, and given that it's a bit of mess, and doesn't seem to have gotten significantly less messy in the year and four months since 4.0's release, I decided that upgrading to Hibernate 4.x was not only unnecessary, but possibly a dangerously bad idea.

What's the Problem?
So, what's the problem, Hibernate team? Why is your documentation not up to date with your latest major release? Why are there significant revisions that seem a little half-baked? Why haven't these problems improved in the 4.1 release and all of its patches? Is there a more fundamental dysfunction that's causing all of these surface-visible problems?

Wednesday, April 3, 2013

Reading 'concurrency-interest'

If you have more than a passing interest in reading and writing concurrent code on the Java platform, hanging out on the concurrency-interest mailing list can be very educational. Occasionally you get to see upcoming revisions to Java libraries, or the mechanics of analyzing a potential change to JDK code.

For instance, this thread recently is an interesting review of CopyOnWriteArrayList's addIfAbsent() method and a potential change to it. Sure, you might not use CopyOnWriteArrayList all the time, and the difference between the two methods might not make a big deal to you, but it's interesting to see some of the kind of discussion and analysis that goes into this kind of a change.


Wednesday, March 28, 2012

Web Development in Pure JSP

On a recent project, my client needed a small web application to be developed in ‘pure JSP’ for a client of theirs.[1] In essence, their only route to deploy anything in the production environment was to commit it to a subversion repository which would later get deployed on their behalf. This made anything but the simplest deployments tricky. Adding frameworks to the project or deploying precompiled code was only going to cause problems.

If you haven’t done much web development in Java, this might not sound particularly unusual. Deploying a php, rails or python project typically doesn’t require much in the way of compilation, and while framework are common to both, doing without wouldn’t be a particular stumbling block.

For a Java project, this is fairly unusual – a classic Java web project uses the MVC pattern and establishes a fairly strict separation between the model, view and controller where the model is plain Java, the controller is a Java servlet, and the view is JSP:


There are lots of alternatives[2] depending on your preferences and technology choices, but I haven’t seen many Java projects implemented using only JSP.

Writing a Pure JSP project would rule out using Java classes[3] for the model and controller, and limit you to whatever you can accomplish in a JSP. Having said that, I couldn’t see any fundamental obstacles to building a project in pure JSP, I just hadn’t done it or seen it done. I did few quick google searches and didn’t see many people suggesting this approach, so I decided to conduct a quick exploratory spike.
After a little testing, I quickly found that there were at least two choices that would work, which I’ll call “JSP Controller” and “Embedded Controller”.

JSP Controller

In this style, you simply replace the controller servlet with a JSP page that acts like a controller servlet. JSP pages are essentially Java servlets in a different form, and if you write a JSP page that contains no HTML or CSS, but instead parses the request from the browser, does any necessary processing and then renders the response using another JSP page, then you have what amounts to a controller written in JSP:


Of course, if you’re avoiding precompiled Java classes, you still can’t use an external Java-based model, but you can use the standard Java class library, as well as inner classes. This isn’t the cleanest way to model your domain, but it’s enough to get by.

Embedded Controller

If your control logic is pretty simple, you might simply embed the control logic in the view up-front. This mixes the view and controller in a way that some might find unpalatable, and a little retro[4], but it does simplify things a little and makes it a little easier to share model code with inner classes. This is the simplest approach, but also the one most likely to make an unmaintainable mess if your project will be growing:


This client project was a very tightly-scoped piece of functionality, so I used Embedded Controller and was reasonably happy with the end result, all things considered.

When to use Pure JSP

I’m not advocating for Pure JSP projects as a generalized solution to web development in Java. It adds a lot of constraints to your development style for benefits that are of limited value in most projects.

But if you need to build a small project that will run in a Java-based environment and you don’t have a lot of access to the production server or detailed knowledge about how it is configured, these approaches do simplify things.

By writing this up, I hope to save some of you a similar exploration; if you’re in this position, you’ll know this is a path that others have taken and that it all worked out in the end.



  1. The project was developed under a non-disclosure agreement (NDA), so I’m going err on the side of non-disclosure here and keep the details vague.  ↩
  2. The model could be an enterprise Javabean; the controller might be plain/annotated Java front-ended by a “front controller” servlet provided by your framework; the view could be Thyme. All three might be replaced by a somewhat different model under JSF or Tapestry.  ↩
  3. Incidentally, JSP pages typically end up as compiled Java classes, but that happens under the cover without your involvement, so it’s a little more suited to an environment over which you have limited control.  ↩
  4. When the web was born, this kind of mixed presentation-and-logic approach was the simplest, and thus, most common approach to adding very simple functionality to websites. In the early days of Java, this was the "Model 1" approach as compared to the "Model 2" MVC separation that came to dominate as a development style.  ↩

Tuesday, November 29, 2011

What Ails Java

Listening to Hypercritical #44: A Little Bit More Sad, about what ails Microsoft, it's hard not to draw parallels to what ails Java.

In particular, John Siracusa suggests that Microsoft:

  • Catered to the enterprise even when that damaged the health of their platform.
  • Failed to capitalize on the strength of their hand by moving the platform forward when their customers had nowhere else to go.
  • Has tended to react to moves made by others rather than being first-mover.
These are tightly coupled with one another, there's a surprising amount of overlap between them, but they each describe some nuance that the other does not.

Each of these has a strong parallel in the history of the Java platform.

Because the control structure of Java is distributed, with Sun, IBM, Apache, open-source contributors, the JCP and so forth, I'm mostly going to talk about Java personified, as if Java itself could have made decisions. That's clearly not true, and it shapes some of that which follows, so I'll talk about that as well.

Catered to the Enterprise
Java developers, as a group, are mostly people who build server-side database-backed web applications for medium-to-large corporations for internal and public use. There are lots of exceptions to the rule in terms of desktop software, non-web-based or non-database-backed server-side code, gaming servers and so forth, but these are a small subset of the overall Java marketplace.

Those guiding the Java platform have set their sights on various markets, from client and desktop software (AWT, Swing, Applets, Java Web Start, JavaFX) to the Web (Servlets, JSP, JSF), media, gaming and the enterprise (Java EE, Enterprise  Javabeans, CORBA), but it's clear that the Enterprise/Web market has always been the stronghold for Java.

Although Java would make attempts at these other markets, they were usually half-hearted and quickly abandoned. It was too easy to continue to emphasize the technologies that mattered to the enterprise and too hard to push into other markets. This also meant that any new announcement about media, gaming, or a client technology would generally be greeted by skepticism by analysts, development companies and individual developers -- each expected that these technologies, interesting though they might be, were probably going to die on the vine out of sheer neglect.

Failed to Capitalize on a Position of Strength
Java started with a number of innovations -- it really popularized namespaces with packages, garbage collection, thorough object-orientation, a virtual-machine-based platform. All of those things existed before Java, but Java really brought those to the masses, made them popular.

However, as Java became more popular, it became far more cautious about innovation, choosing to favor backwards compatibility over significant change. This caused the language to settle down, to stagnate or mature, depending on how you feel about the results. It became a language in which you could get things done, but a language about which one might cease to be excited about.

The runtime started to take on bloat as things would be added (JavaDB, CORBA), but never removed, no matter how little they were used. There has been some talk about modularizing the JVM, but it's only talk, and too little, too late at that.

In order to ensure that conservative development organizations could upgrade without having to think about it, classes and methods would be marked as deprecated, but never pruned.  As a result, you could start to tell when an API was created by the age of its dependencies. For instance, HttpServlet in Java Enterprise Edition 6, delivered in 2009, still uses Enumeration despite the fact that it has been essentially replaced by Iterator in Java Standard Edition 1.2, delivered in 1998, eleven years earlier.

It didn't need to be this way. Throughout much of its history, Java was in a position of strength with its primary customer-base, the enterprise. If the guiding forces behind Java had said, "Ok, this has been deprecated for two major versions, let's get rid of it now" I'm entirely sure that their most cautious enterprise customers (e.g. major banks) would have grumbled and complained, but gone on to modify their code to remove Enumeration, Vector, Date.va getYear() and other similar pieces of outdated Java code.

Sure, it's a pain to update your code on a mature program to reflect changes in the platform, but think how much more of a pain it would be to port that mature program to .NET, let alone Ruby or Python. Where did Sun and Java think these enterprises were going to go if they pushed back a little?

Java could have capitalized on this strength and taken the opportunity to move the platform forward even when that meant some grumbling from the enterprise customer base, because the cost of switching was too high. This has to be done carefully -- you can only push so hard without alienating your customers, but Java didn't push at all.

Reactionary
Because Java has catered to its enterprise customers and failed to capitalize on a position of strength, the rate of innovation dropped off precipitously. This put Java into a reactionary mode where changes were only made as perceived threats grew stronger and reacting to those threats became vital.

This means that for years, Java hasn't been the innovator, but simply introduced changes to stem the bleeding caused by other languages and platforms innovating around it. Look at the rate of change to the C# language in the same timeframe that it's overlapped with Java. Look at Java continue to struggle with generics and closures, let alone platform modularity and aging libraries.

People don't look to Java to innovate anymore, and that means they're not excited by Java, and as they get excited by other languages and platforms, their attention wanders. Java has been ripe for disruption for years, and sooner or later another platform or language is going to make that happen.

Java's Control Structure
Java's distributed control structure and odd profit models have encouraged much of the above. Sun, Oracle and IBM, key figures in steering the Java platform, are all large businesses targeting enterprise software development. They all make most of their Java-related money from consulting rather than selling tooling, or direct sales to end-users.  So it's no real surprise that these organizations would choose to cater to the enterprise. 

The control structure is also very distributed, with a committee based on these organizations and individual expert contributors. This kind of control structure is good for getting consensus, but is often cautious and rarely innovates. That tends to result in making cautious moves, rather than bold ones that might capitalize on a position of strength, and this tends to lead to a reactionary approach.

Ultimately, you could probably argue successfully that much of what ails Java are a direct result of its control structure and participants. This structure is very different than Microsoft's, although you could probably also argue that their problems are structural.

In Summary
Java, like Microsoft, has not aged gracefully. As each got popular, it also got more cautious and settled into some long-term patterns that guaranteed that it would please its primary customer-base, the enterprise, at the cost of long-term health. Both are in decline and ripe for further disruption.

Neither is likely to go away any time soon -- they both have their hooks in deeply with the enterprise, which will hang on to avoid the cost of change, but both have their best days in the rear view mirror and can only hope to age gracefully, rather than find new life.

In Java's case, there's some possibility that another language will find strength while building on top of the rest of the Java platform, in particular the virtual machine. I suspect that even these are simply stop-gap measures while the libraries attached to the JVM continue to decay, but that's the subject of another post.

Ultimately, Java is still a perfectly solid environment on which to build enterprise software, but it's now the safe choice, not the interesting one. That's been true for years, and it's only a question of time before some other language and/or platform finds dominance over Java and increases the rate of decline.

Monday, January 17, 2011

Jr. / Int. Developer

A client of mine, TPA Outsourcing, is looking for a Junior or Intermediate Developer to join the company. It's an interesting position in that you'd be their first full-time development employee, which means that you'll be very involved in how the company's software evolves, and there will be lots of opportunities for growth.

In essence, TPA has already built one software product, and they have plans for others, and while I believe they're happy with the way the work has progressed, they are aware that they need someone on-board full-time who understands these systems, can continue to evolve them, support ongoing growth, and so forth, and that the sooner they get someone on board who can become their expert in these systems, the better.

The developer that joins TPA would work with an experienced team right away to enhance one and build other software products, become expert in those systems, and take on more responsibility as time goes on to become the primary development role for the company as time goes on, plan future products and enhancements, work with TPA to decide if and when to build a larger software team, mentor new team members on existing products, and so on.

While we value broad-based development experience and passion for software more than we do specific technical skills, it's worth noting that the existing product is built using Google Web Toolkit (GWT), Java, Hibernate & JDBC, built with Maven and is tested using JUnit & JMock. If you've got experience in those areas, that can only help.

If you're interested in talking further about the role and how you might fit the position, I'd be happy to discuss it more detail. Get in touch.