Tuesday, January 29, 2013

Iterator is a design pattern

Iterators are used so frequently that they have become implicit in modern programming environments, with foreach loops replacing explicit getIterator() and next() calls in typical designs.  It is therefore somewhat strange to think of iterator as a design pattern - it is so simple, common and abstracted that one hardly thinks about iterators as anything other than an implementation detail.

This got me to thinking about the nature of programming, as our tools and environments continue to evolve.  I was looking at a problem recently of adding logging to a number of functions in a C# application and my mind immediately jumped to implementing the feature using a decorator, as I would if tasked with the same problem in Python.  Django development has pushed the decorator pattern into the same category of common and trivial code as iterators in my mind, and yet decorators remain a far less standard programming construct across other languages and frameworks.  This is not to say C# doesn't exhibit a similar relationship with other design patterns - WPF databinding has likely antiquated thinking about the observer pattern for many front end developers.  Today, we stand on the shoulders of giants when starting any new application and simply need to choose the best fitting option.

As languages and frameworks continue to evolve, I look forward to seeing how this process continues.  Perhaps worrying about making an application RESTful will one day seem as trivial as iterators.  Wouldn't that be spiffy?

Wednesday, January 2, 2013

My contribution to the excessively obscure

Eclipse is an amazing program and a triumph of the open source community.  I genuinely think very, very highly of it.  That said, I have to admit that I sometimes suspect they have a slightly sinister marketing plan, where every so often the plugins destroy themselves or one another, just so the end user will look through their plugin list and appreciate all they've done with Eclipse over the years.

In my case, my plugin list contains a veritable who's who list of add ons - Google's Android plugins, Amazon's plugins for AWS, CDT for C++ development, J2EE, both CVS and Git integration, my beloved PyDev plugin and a handful of others.  Somewhere along the way, something didn't play well with something else, and I lost several editors.  Following the generally safe advice from "some guy on stackoverflow" I ran eclipse -clean, which mostly just broke things in a new way.

Reading through forums and mailing lists, most roads lead to advice to just wipe everything and start from a clean install.  Fortunately, with a bit of tinkering, I managed to restore my Eclipse to working order, and that's the cause for today's post.

Every so often, I google for an absurdly specific problem and just as I'm about to give up hope, miraculously find someone, somewhere has documented the exact same issue.  Every Linux user has had this experience at some point I'm sure.  In the off chance someone ends up here, this was my excessively obscure problem and solution:

On Eclipse 3.7.2, running under Ubuntu 12.04, with Eclipse Java Web Developer Tools 3.3.2 installed, installing the Web Page Editor 2.3.6 appeared to have no effect.  It showed as installed, but the editor did not appear in any menus.  Attempts to uninstall and reinstall, run eclipse -clean and check for plugin updates were no help.  Uninstalling both plugins and reinstalling only Web Page Editor solved the problem.  A fairly simple fix to try before going the full reinstall route if anyone ever finds their system in the same state.

Wednesday, December 19, 2012

A hammer for a nail

I've viewed Test Driven Development with some skepticism for a number of years.  TDD is sometimes preached as a core tenet of software engineering, but I've long held that it only makes sense in specific environments, such as web applications.  I'm also a fervent opponent of 'vanity metrics' and I have yet to see a project where lines of code covered by unit tests translated into real world improvement in product performance or developer productivity.  Unit tests are a great tool for developers, but when they become mandated it's too easy for developers to focus on meeting the build or integration criteria and stop thinking critically about how the code might fail.

Which is why I'm happy to say that my current project is very much following Test Driven Development principles.  TDD fits perfectly for Django application development.  The final product will be a heavily AJAX web app, and yet I've been able to design and test with confidence all the server-side logic without writing a line of javascript.  More useful still, as a developer relatively new to the Django framework, the suite of tests have made my multiple redesigns go much more smoothly.

I'm looking forward to seeing how my development process continues to evolve as my solo projects move forward.

Tuesday, December 11, 2012

The surprising cost of doing nothing with Celery and SQS

I'm a big fan of leveraging open source solutions, so Celery seemed the obvious solution when I needed an asynchronous task scheduler for a Django project.  Since I was hosting on Amazon EC2 and on a tight schedule, I opted to use Amazon's Simple Queue Service (SQS) to save time, rather than setting up RabbitMQ on my server.  My use case had a trivially small number of messages being sent, so I naively assumed the advertised "First 100,000 Amazon SQS Requests per month are free" would more than cover my needs, and quickly moved on to more pressing tasks.

I had a billing alert set at a single cent, so Amazon emailed me as soon as I had gone over 100,000 requests and started being billed for my usage.  This happened quite quickly, while I was still in the early stages of development.  A quick check of the AWS console, the terms of SQS and the Celery documentation made my folly clear - by default Celery will spawn workers for each core with a default polling rate of 1 second.  My quad-core development machine, running a local django server instance but accessing SQS had been polling an empty queue 4 times every second, meaning I burned through 100,000 requests in a single day.

Celery can easily be configured to change the poll rate and number of workers, and SQS is only 1 cent per 10,000 requests as of this writing, but it's still a silly way to waste money.  The experience did make me take a moment to reconsider my design, because once the site was sending messages and using Celery's periodic tasks as well, I realized my usage of SQS was going to be more expensive than originally expected.

Friday, August 3, 2012

I'm now working full time on residence life management software at Wosga.  We're looking to provide easy to use tools for resident advisors, residence directors and school administrators to remove paper waste and increase staff efficiency.