Paying technical debt dues

The Texas Tribune didn’t just hire me to play third base on its softball team, although my performance in the absence of the injured photo editor definitely helped keep me in Austin for the past year.

My primary responsibility was to rebuild the government salaries website, an ambitious project to open the compensation information of Texas public workers by sending lots and lots of open records requests to countless state agencies.

Gender and ethnic salary analysis were new features of the rebuilt salary app.
Gender and ethnic salary analysis were new features of the rebuilt salary app.

Former Tribune developers built the original site in 2010 and was received much like other salary sites. It generated so much traffic and criticism from the individuals represented in the data that publisher Evan Smith had to write a blog post.

Why rebuild such a popular app? Technical debt.

The salary site was supposed to be updated when Tribune staff received information from their records requests, but the app fell woefully out of date because importing data was complex and time consuming. Tribune staff paid in the price in daily angry emails from state workers who were upset that anyone could find their outdated salary with a Google search.

Government Salaries v2.0 became my primary task in February. The MVP had three goals:

  1. Reduce technical debt
  2. Reduce angry emails
  3. Provide new analysis

Most of my work on the six-month project focused on importing and storing data. The new app used a rebuilt import process, an open government data schema championed by Travis Swicegood in his ONA talk and Docker for deployment. I looked to metrics, wireframes and user personas for design guidance and tested features as prototypes with the newsroom.

My paper prototypes looked rough.
My paper prototypes looked rough.

I was disappointed that I did not provide users transparency into the development process, a concern that was raised at the Hacking News Leadership conference in Austin. The status of the salary site is of great importance to state workers, and a development blog explaining our progress would have answered many emails and phone calls.

I moved to Los Angeles to be with my partner before the new salary app launched, but Ryan Murphy, Daniel Craigmile and Alex Duner did a great job carrying it to the finish line. When the app went live Thursday, I was both excited for users to finally see new data with gender and ethnic breakdowns and stunned to see months of work published.

Of course the project is not complete. In addition to updating the data, I hope my code and documentation help future developers add features and provide insights into Texas public agencies. I understand the relaunch generated more than 200 emails, and I wish those developers well in responding to them.