Wednesday, April 11, 2018

Managing SSH Whitelists on AWS with awswl

If you have EC2 instances on AWS,  it is common for them to be layered behind firewalls implemented with VPC Security Groups. That means that if you need to access these servers directly, you may not be able to unless you take measures to make that happen.

In an enterprise AWS account, there are lots of good solutions to this problem. Firstly, if you are far enough down the containerization path, you may argue that directly accessing the instances via SSH  is to be avoided, that you should only be building and deploying containers.  Alternately, if you do need to access these servers, you can likely do so with a VPN, either hardware or software.

However, for a smaller AWS account, like a small project or small business, these solutions may be more complicated than you desire. I find myself needing to access small AWS accounts from a variety of places, as I move around a fair bit meeting with clients and working, and I need to be able to access EC2 instances on AWS while I do, so I found myself wanting a tool that would allow me to quickly add and remove my current external IP address or particular networks (expressed through CIDR blocks) to an AWS security group.

So I built a little open-source tool. Since the most-popular AWS client library is boto, and since python is a reasonable choice for a simple cross-platform cli tool, I built it in Python. I called it awswl (aws whitelist) and I've been refining it, adding tests, documentation, making sure it works with both python2 and python3.  Now it's finally ready to release to the wild.

You can find it on pypi if you want to install it, on GitHub if you want to read the source or contribute, and you can browse the documentation on either one.

Thursday, April 5, 2018

Automatically Add Taxes to Invoice on New Freshbooks

I migrated to the "New Freshbooks" almost two years back, I think -- and for the most part it went fairly smooth. The design is a little cleaner, although there are a few features of the old system that aren't present.

Initially, when finding a feature gap, I assumed it was coming. Some of those features, however, still haven't come, and I'm tired of waiting. In particular, the ability to add taxes automatically to all the lines of an invoice is a real thorn in my side because for some of my clients I have very detailed timesheet invoices.

On the latest example of this, I had 87 lines. Going through 87 lines one by one and clicking "Add Taxes", then a checkbox, then save is ... repetitive, boring, and just the sort of work that computers are great at and humans are less great at. And yet I've done this on a whole ton of invoices because there wasn't an automatic way to do this on the new FreshBooks.

But no more. I gave up waiting for FreshBooks to do it and wrote a little piece of JavaScript that I can run with Tampermonkey:

// ==UserScript==
// @name         Add HST to FreshBooks Invoice
// @namespace
// @version      1.0
// @description  Go through a FreshBooks Invoice and add HST to each line
// @author       Geoffrey Wiseman
// @match*
// @grant        none
// @run-at       context-menu
// ==/UserScript==

(function() {
    'use strict';

    let popovers = document.querySelectorAll("div.js-tax-picker-popover");
    for( var popover of popovers ) {
        let hst = popover.querySelectorAll("td.js-taxes-popover-checkbox")[1];


    alert( "Done adding HST to Invoice." );

Now I just edit an invoice, right click, select "Tamper Monkey" and then "Add HST to FreshBooks Invoice" and wait for it to run. Way, way better than three clicks per line.

If you have the same problem, feel free to adopt my solution. You might need to adjust the index of the popover checkbox if you don't want to add the second-defined tax (I used to have PST and GST), now I just have HST -- but other than that it should probably work for you.

Monday, January 29, 2018

ServiceWorkers coming to Safari

If ServiceWorkers are coming to Safari for macOS and iOS, it looks like progressive web applications will be pretty plausible across all major modern browsers -- both desktop and mobile. You'll still have a problem with older browsers (and thus older machines that haven't updated), but this can still get you a single codebase across desktop and mobile, which is appealing to a lot of companies. It'll be curious to see how much of an impact this has on React Native traction.

Thursday, January 25, 2018

Configuring Firewalld on CentOS 7 with Ansible

I've been working on the configuration of a new server with Ansible; this client has CentOS servers, so I'm configuring it for the current release of CentOS 7.x, which comes with Firewalld. All of the existing playbooks are set up for iptables, which was used in earlier versions of CentOS.

I thought about rolling it back to iptables, but I decided to try using firewalld first. I hit some problems:

  • Firewalld port forwarding only supports remote traffic:
    • If I want to run Tomcat and use the firewall to forward from port 80 to the Tomcat port, then accessing http://localhost will not trigger these port forwarding rules.
    • Fortunately, you can work around that with a "direct" rule.
  • The ansible firewalld module seems immature:
    • flagged as 'preview'
    • doesn't support port forwarding
    • doesn't support direct rules
    • You can work around that by invoking `firewall-cmd` using the command module.
  • Not easy to use `firewall-cmd` in an idempotent way.
    • Firewalld is configured with commands, somewhat like iptables. You can run these commands using the command module.
    • It's not that easy to invoke these commands in Ansible in a way that lets you be properly idempotent -- only run this command or mark it changed if something has changed. As a result, these will run every time, and if you have a handler to restart the firewalld service, that will also trigger every time.
What I ended up doing is configuring firewalld with commands, then looking at the configuration files that result and instead of having the ansible playbook trigger these commands, I have the playbook copy these files into place. File copying is something that is easier to do in an idempotent way than command invocation, so I can configure Ansible to copy configuration files into place (/etc/firewall/direct.xml, /etc/firewall/zones/public.xml), and then restart firewalld if the files have changed.

That seems to be reasonably happy.

Friday, May 26, 2017

Limiting Android Localization to Supported Languages

When you're working with an Android app that will be used in a variety of markets, localization is an important feature to support the different countries, date formats, number and currency formats that are used around the world. Android has deep support for localization which you can read about in great detail.

In iOS, in order to add support for a language, you add languages to your project properties in an explicit way. In Android, adding support for a language is more implicit -- you simply start localizing resources. This is easy to use but it creates a problem where you might have partial support for a language in place but not be ready to take that support live. Android doesn't offer a means of explicitly listing which languages you wish customers to have access to. This can lead to situations where you have to keep localization resources on a branch or out of source-control or otherwise limit how those files are managed. That's viable but it isn't a great fit for every workflow.

On a recent client project, I did a little work with Gradle to make supported languages more explicit.  I added a list of supported languages in the build.gradle, used resource shrinking with resConfigs and exposed the languages to Java in BuildConfig using buildConfigField:

// Approved Languages
def approvedLanguages = [ "en", "ru" ]
resConfigs approvedLanguages
buildConfigField "String[]", "APPROVED_LANGUAGES", "{\"" + approvedLanguages.join("\",\"") + "\"}"

By exposing the approved languages to Java, it was possible to use it localization code, for instance to only localize dates to languages that are in the supported language list:
public class LocaleUtils {

     public static boolean isSupported(Locale locale) {
        for (String language : BuildConfig.APPROVED_LANGUAGES ) {
            if (locale.equals(locale.getLanguage()))
        for (String language : BuildConfig.APPROVED_LANGUAGES) {
            if (language.equals(locale.getLanguage()))
                 return true;
         return false;

     public static String localizeLocalDate(LocalDate localDate, String format) {
        return localDate.toString(
        Locale locale = getDefaultSupportedLocale();
        DateTimeFormatter formatter = DateTimeFormat.forPattern(format).withLocale(locale);
        return localDate.toString(formatter);

With this in place, we can start working on a new locale, integrating translated strings and graphics in the main line of development and know that they won't be exposed to customers until we're ready. With a little work, we could also use a different list of supported languages on a debug and production build.

Monday, October 24, 2016

Swift Mutable, Hashable and Collections

Swift sets have similar behaviour to Java sets when it comes to mutable objects: if you have a mutable object, wherein mutation of the object can result in a change to the hash value, that can lead to surprising behaviour. The collection may store the object in a bucket based on the hash it had at the time of insertion. If you modify the object, the hash value changes, but it doesn't move from one bucket to another. This can mean that a set may report that the object is not present even when it is.

I made a little playground (raw view, compressed form for download) to demonstrate this issue, but in essence:

let mi = MutableInt(1)
let mis:Set = [mi]

mis.contains(mi) // true
mi.value = 2
mis.contains(mi) // false

The same object is present in the same set for both contains calls, but because the hash value has changed, the set can no longer find it.

Sunday, September 18, 2016

Host Key Verification and Ansible

I've been using Ansible to configure some instances on Amazon Web Services with a client, for two reasons:
  • To make it repeatable in the future, if we want to configure an instance again, or another instance.
  • Because I will be repeating it immediately now, making more than one server to live behind a load balancer that are meant to be configured identically.
When you connect to a host for the first time over SSH, you are asked to verify the host key. When you use Ansibleagainst a host for the first time, the same thing happens. If you are connecting to multiple hosts at the same time, bad things happen -- you get multiple prompts to verify the host key and responding to them doesn't seem to work:

$ ansible servergroup -m ping
The authenticity of host ' ()' can't be established.
ECDSA key fingerprint is SHA256:hEdMy3XKWV/zWobmSuwf+b6oI9xt4cYJzM1eAa2T8Ak.
Are you sure you want to continue connecting (yes/no)? 
The authenticity of host ' ()' can't be established.
ECDSA key fingerprint is SHA256:N5iv0/+zRHk7UTsIQOUlzn2ZiU9L2xL+Fn153nlZdjs.
Are you sure you want to continue connecting (yes/no)? yes
Please type 'yes' or 'no': yes
Please type 'yes' or 'no': yes
Please type 'yes' or 'no': yes
Please type 'yes' or 'no': yes
Please type 'yes' or 'no': yes
Please type 'yes' or 'no': ^CProcess WorkerProcess-3:
Process WorkerProcess-2:
Traceback (most recent call last):
 [ERROR]: User interrupted execution

 While I was pleased to read the post about bugs filed, it doesn't seem to be fixed yet.

Happily, as long as you connect to a single host at at time, everything is fine, and after that you can connect to the group:
$ ansible server-one -m ping
The authenticity of host ' ()' can't be established.
ECDSA key fingerprint is SHA256:N5iv0/+zRHk7UTsIQOUlzn2ZiU9L2xL+Fn153nlZdjs.
Are you sure you want to continue connecting (yes/no)? yes
server-one | SUCCESS => {
    "changed": false,
    "ping": "pong"
That's fine for a couple of servers, but it wouldn't be fun with a large cluster. If there's a better way, I haven't discovered it yet.