iTools: Building A System; Part 1 – The Dilemma

Greetings to everyone out there, whether in the countryside on a farm, in an office in the city, or somewhere in between. The past two weeks have been an interesting time at Grown in the City. The food sovereignty movement in Maine has sparked high interest from communities across the country – well beyond the borders of that state. When Grown in the City first reported on the story, we knew it was important and unique, but by following it with interactive tools for others to use, we’ve begun to create a network of people who are passionate about this (and other) topics. The experience of getting such a resource from idea to useable product has caused us to think hard about how we want to collect and deliver data, in an effort to support movements that are much greater than ourselves.

In this five part series, I will break down what we did to provide an interactive mapping system that we’ve since used to create food sovereignty, urban agriculture zoning, and food policy council maps.

In part 1, I will discuss a dilemma that all those working with data face when they look to share it. Finding an answer was crucial to determining how we wanted to execute our ideas, and influenced what the project eventually became – an interactive, open source map. In part 2, on Tuesday, I will discuss the data itself. Wednesday’s installment will talk about getting stored data sets to the front end of the map. On Thursday, in part 4, I will discuss how to push stored data onto a map for everyone to see. Finally, in part 5, I will discuss how we styled the data on the page, to give it a visually attractive look, and keep it consistent with the rest of a website.

The topic may seem complex, but I will try to keep it as simple as possible for those without any tech background, while making it descriptive enough for the tech community to understand and implement. My goal is to have you understanding how the process works by the end of this, and I hope that some of you might even challenge yourself to create a map or data set that is useful to your readership.

Part 1 – The Dilemma

When I started working in the computer industry 12 years ago, dial-up was still prevalent, and dynamic websites were just getting started. We were building large systems – applications used by large corporations to process one task. This required large databases, normalization, stored procedures, database backups, and so much more. We built the equivalent of a football stadium for a team to play in, every time. Fast-forward 12 years, and the technology world, which has fallen into the mainstream just as heavily as the television did forty years before, is all about little things. “There’s an app for that” is the new mantra in this world. You want to do something small? Make an app, it does one thing, it does it well, and it costs 99 cents to own. If you look at the iPhone model – it is thousands of small apps, meant to do one purpose, but combined, you now have a full system – for whatever you want to do.

My IT world had changed. I knew it was happening, and was up to date in my home technology, but in my day job, I was still feeling like I was building monolithic systems that do everything but brew coffee. In the past few months, however, I’ve been working with John Reinhardt on small freelance projects. My days of drawing up a 2 year long scope of work to create the best large-scale application just got turned upside down. I would take six months to build a website, while he would use the WordPress platform, and have a site up for a client in a week. Did he understand the underlying technology used? No, not entirely. Did it matter? An even bigger “no.”

So, why does this matter? This is where it got interesting. John came to me one day with the idea for maps that would store, organize, and display information pertinent to this blog, Grown in the City.

Should I house data in SQL Server, my favorite Microsoft product ever (sorry Xbox!)? Or in Oracle and get my command line on? Should I do it on a Linux box? Sounds fun! Or should I go with MySQL, do the freeware thing, access it from a Java-based client on a hosting site. Now we’re talking right? Build some normalized tables with referential integrity, stored procedures for data access, and put my DBA hat on! But, wait, Johnny (and many of you out there, I imagine) don’t understand any of this, so then I would have to build a front-end application just so he can access and update the data.

Our simple little map system was now a four week project, with hours of coding, research, and training. This was going to take a while. One weekend, John was working on the site and said, “send me the code for a map.” I sent him over to the Google Visualization website, and two hours later he has the shell of a map up. Did it do everything we imagined? No. Was it pretty? No. Was it serving its purpose of providing fellow colleagues and enthusiasts with important data? A resounding “Yes!”

From that initial map, we set out improving the system, and built the products we have today.

So, as for my dilemma? Well, quite simply, I understand too much. I know how to optimize indexes, create custom user-defined functions, and hand-code applications from scratch. But can I get something up in two hours — to actually help people? Well, this was the unique challenge. It turned my skills upside down, making them work for me in real-time, rather than in strict theory.

The whole experience taught me that sometimes, simple is beautiful. We ended up creating an open source map system that even John was able to understand. And sometimes, just because you know how to do something complicated, it doesn’t make it the best solution! Over the next few days I’ll be explaining just how we did it.

Check back tomorrow for Part 2 of the series, where Bob will talk about the data itself.

This originally appeared on Grown In The City.

_________________________________