Persistence, Processing and Presentation in the Cloud Based Applications

I’m sure several things come quickly to mind when someone mentions cloud computing or cloud applications. As Mark noted in his post about different emerging cloud services there are several layers of complexity that might flavor an application built on or with cloud services. I’m going to specifically talk about building applications that consume or consist of cloud services, Software as a Service ( SaaS ). These can be services you build yourself or they can be 3rd party services. At any rate your end goal would be to have an application that can be delivered to your users, built upon cloud services and my goal is to describe the different layers of how you can build that application based on my experience of designing/building cloud based applications.

I tend to think of the cloud just as I would a traditional computing solution with a few caveats: cloud computing should be highly scalable, on demand and more cost-effective than something I could get at my local web hosting company. At least in theory. In five years time, what we think of as cloud computing will probably be the “traditional” computing solution, indeed you will notice that the layers I’ll describe already follow how web applications are generally built today. The reality of the situation means that while cloud computing solutions offer more for less, the trade off is that no one service can be 100% trusted, no one network can always be relied upon, and therefore fault tolerance must be built into the system.

So let’s break down the anatomy of a typical web based cloud application. Generally speaking applications fall into 3 layers, 1) Persistence Layer, 2) Processing Layer and 3) Presentation Layer. These layers can consist of several different services, from several different cloud computing vendors or api’s. You should feel free to have all of these layers talk to each other as well.

So, for the Persistence Layer you might write to a EC2 hosted MySQL database for app specific data, while also relying on the the Persistent capabilities of Facebook, Twitter, or even Google to store user specific data. When I think of the the Persistence Layer I think of only reading and writing data to disk. There should be no (or as little as possible) logic associated with this layer because any logic you write will tend to be platform specific making it more difficult to port in the future event you have to switch providers. In most cloud computing solutions this layer is the layer that implements Eventual Consistency. Interactions with this layer should be quick and to the point. You write, you read. Period. It doesn’t happen often, but if you were to move your persistence layer from one vendor to another, it should be a simple as transferring files.

The Processing layer ( just like a traditional application ) is the layer that interacts with both the Presentation and Persistence layers. This could be a layer you write in your favorite server scripting language. It could be a proxy layer that you write that takes input from your application and then routes data to other third party services, with some of it going/coming from Amazon SimpleDB, some of it going to a legacy database from with your network, and perhaps it ferries analytical information to Google Analytics or Omniture. The point is that, the processing layer can be used for any type of processing that your application needs. But keep this in mind, in my experience the more you proxy to third party services, the less code you’ll have to write and maintain. Or to look at this from the other side if you’re using a third party API and your application doesn’t need to augment the data before sending it over there should be no reason to build a processing layer. Mashups can be thought of a cloud application without the Processing Layer. Can your Presentation Layer talk directly to Omniture or Amazon S3? Absolutely, as long as you don’t have application logic that needs to run before you read or commit.

The Presentation Layer is where everything gets tied together. Above all it needs to always be responsive. I’d go so far as to say that it’s development specific limitations should drive all other development, when possible. I realize that might be a bit controversial, but remember in most use cases, the Presentation layer is running on a user’s machine, which can be thought of just another machine in the cloud. The user is allowing us to run the application on his/her processor for free with the implicit expectation that the application will give something back. The very least we can do as developers is to make sure the user knows what is going on. The need to pull data from multiple locations and present in a data rich interfaces is what makes technologies like Adobe Flex and Microsoft Silverlight so appealing to developers.

So, as a quick summary about what to think about when building cloud based applications. Build the Presentation and the Persistence Layers first and only add the Processing Layer as needed. Never trust any one server or any one network. Practice failure to help build Fault Tolerance. And last but not least, the most important fault tolerance you can build is on the Presentation Layer.

In posts to follow I’ll take a deeper look at each of these layers, various options for these layers and talk about some the lessons learned and best practices for building applications upon cloud services.