There is an undoubted need for a hybrid architecture for many larger corporations since migrating existing apps to the cloud is not a simple as lot of demos show and there is a perception (whether real or not) that the data is less secure on the cloud. Enter hybrids apps – maintain the data on premise or consume on-premise apps from a cloud service.

Of course it is possible to communicate between on-premise data sources or apps and cloud-based apps using SOAP/REST communication protocols, however there are two major obstacles – discovering the service endpoints (since these may change due to dynamically assigned IPs)  and navigating through firewalls. These problems can be overcome by allowing apps to selectively open ports which is inherently insecure, and using relay systems that sit between the firewall and the apps and act as a bridge, thee systems tend to be very complicated and hard to implement.

The Azure Service Bus attempts to solve this issue by proving a service which allows applications which need to communicate with eachother to register with it. The requesting app is given a Service Bus endpoint to communicate with the data source/service app. Essentially the services are provided by service apps run behind the firewall, and the connection endpoints are provided by the Azure Service Bus. It should be noted that the Service Bus allows communication with non-.NET services , so Linux/UNIX hosted apps can register with the Service  Bus and be consumed by .NET apps.

Security is provided by the Azure AppFabric Access Control, which applies user-defined rules to ensure security when an app claims tokens via the STS service provided by the Access Control.

Thus the Service Bus can be used to build hybrid apps which span both on-premise and cloud services.

Adoption rates in future is, however, an interesting question and the answer really depends on who you talk to you. The staunchest critics seem to be DBA’s and SysAdmins who are used to the fine control they have over the hardware and the software stack. Moving to Azure surrenders most of this control to Microsoft and in truth wouldn’t leave DBA’s and SysAdmins a lot to do. In the opposing camp are small developers who have limited resources but need to deliver and maintain apps and, in common with most developers, have little idea about hardware, security or scaling applications. For these developers Azure is an ideal platform, as opposed to traditional ASP.NET Hosting since it takes most of the admin, security and application scaling chores away and lets users focus solely on building  the app. A bonus for web-app start-ups is that the hurdle of persuading users to trust the app with personal data, when the app is hosted on Azure it is a powerful marketing tool to highlight the platform’s enterprise level security.

Another compelling use case are apps which are of variable scale. Typically hardware must be set up to handle the maximum load an app will be under, this can be very tricky when the max load is vastly different from the typical load. At PDC Microsoft a ticketing app was demonstrated which was suddenly under intense load when new tickets were made available for popular sports events but under minimal load thereafter. The ability of Azure to scale effortlessly up and then down makes it a very natural fit for these apps. Large apps which need a server farm are a less compelling case, Azure certainly simplifies the scale but facilitates very little in the performance fine tuning of the scaling process. In addition it is definitely cheaper to run the apps on a server farm than Azure (ignoring any saving on admin personnel that using Azure may allow).

2010 is likely to be a defacto beta year for Azure its real test is likely to come down the road.

This marks a major shift in Azure’s focus, previously Azure was intended to perform all hardware configuration chores from developers. SearchCloudComputing attributes the change to easing the migration pains for existing Windows apps, however resource isolation and allocation seems a much bigger step forward. A lot of developers who are new to clouds are surprised to find that their apps are not fully isolated from other cloud apps. For this reason Azure imposes a lot of restrictions on apps – for example the execution time of a SQL Azure query is limited to 5 minutes after which the connection is terminated and must be re-established. A VM would allocate a user a finite amount of resources to use as that app wants thus removing resource throttling.

No firm release schedule or other feature details were announced.

Enter the Azure cloud, now .NET developers have an arsenal of API’s to manage and scale an application. Setting up a production environment takes a few clicks and all the infrastructure is fully managed. Seems a wonderful promise, but what it the reality?

I waited until October to kick the tires of Azure, I normally only use software in Beta 2 as I’ve spent too many hours trying to figure out an issue only to find that it is a known bug (which is nevertheless undocumented) and is fixed in the next release. So here’s my first impressions:

• Sign-up process is just terrible. Long story short – you need a a Live ID (or hotmail account as we normally call it) you are then forwarded to Microsoft Connect (which I never heard of before and seems to have little to do with Azure), the Connect site crashed on me several times, the account status on Connect was listed as ‘participating’ in Azure despite the fact I still had to wait for my activation key and no email was ever sent to confirm the registration. Looking through the forums, I am definitely not alone on this and I hope this improves as Azure matures.
• Web interface is a little sluggish, but quite intuitive and easy to navigate. I would have liked to see more AJAX used as there are a lot of page refreshes. In future, I’d hope to see a slick Silverlight interface.
• The SQL Azure web interface is extremely limited, just create and drop databases, collect connection string and set up firewalls. Thats it. I was hoping for a much more extensive set of online controls such as managing users and logins. This functionality is of course available programmatically and through SSMS (see below) but some common functions on the  UI is always useful as well.
• No SQL Azure backup ! As far as I am concerned this is the biggest weakness of Azure at launch. Azure’s promise to replicate the data at least three times protects against data loss due to hardware or systems failure, but a key role of backup is the ability to roll back after an error (such as accidentally dropping the entire database!). And for this, Azure provides nothing. There are workarounds of course such as BulkCopy or BCP, but Azure should provide this out of the box. Backups are promised in 2010 and it is difficult to make a strong case for Azure until then.
• Encryption and compression are also absent but promised for 2010.
• Azure Data Storage looks great – Blobs, Tables, Queues and Drives. All with simple API’s to manage them- saving and loading a Blob takes just a few lines of code. Still quite a bit of flux in finalizing the Storage API’s for Azure’s launch in January 2010. The API’s are not included as standard in the Visual Studio project templates and you have to build the Storage dll and then reference it in your project.
• Tables! Definitely my favorite feature of Azure right now. Ever thought setting up a database for a simple task like handling a few users and their comments was overkill? Tables are perfect for this sort of semi-structured data. Simple to setup and interact with and very fast, more importantly they are free.
• Tables are my favorite feature but the CDN comes a very close second. Azure comes with a built in content Devliery Network for blobs. CDN’s ensure that large files are served from their closest location thus reducing the latency that end users experience requesting large files from a very distant server. The Azure CDN currently has 18 geographic locations to server the files from. Configuration is a breeze (just change the URI of the file to be accessed). Caching is supported and there is also the option to register a custom domain name. Pricing for the CDN will be additional to standard blob storage but there as been no announcement as yet.
• The local development environment provided in the Azure SDK works very nicely but there is no local version of SQL Azure. You have to use a local SQL Server for testing which is far from ideal as there are a lot of incompatibilities between SQL Server and SQL Azure (such as User Defined DataTypes not supported in SQL Azure). It is very easy to connect to an Azure database, simply substitute in a SQL Azure connection string, but this isnt really the point what we’d like is a full local environment for testing. The only real solution now is a local Windows Azure environment connected to an online testing SQL Azure database.
• SSMS can connect to SQL Azure databases but it is far from full featured as it doesnt support viewing objects from the Object Explorer so lots of TSQL is needed to work with the database. There are some third party tools available (such as SQL Azure Manager)  which go some way to plugging this gap.
• Deploying apps is a snap, Visual Studio rolls the app into a package file which is the actual application and a configuration file which are the configuration settings. These are both uploaded from the Azure developer portal and the app can be moved to production with a few clicks. Apps which use Windows Azure Storage have an additional deployment step as Storage requires separate provisioning in Azure but the process is still very smooth.
• Migrating existing apps is another story. I admit I didnt dive into this in great depth but migrating an app is not going to be painless. For starters there are the SQL Server features that Azure doesnt support, there is an Azure migration wizard on CodePlex but it won’t be able to account for some inconsistencies such as the USE statement not being support in SQL Azure. The new API for logging,  accessing files is now being handled by  Azure Storage and just two of the issues that existing .NET apps would have to handle in a migration. It seems Azure will be mainly used for running purpose built apps and not for running existing apps.

SQL Server and SQL Azure are great products but for a lot (maybe even the vast majority) of cases they are overkill and add an unnecessary overhead and slow development. But there are almost no alternatives to persist structured data in a .NET application. XML is a good choice for small data storage tasks or for transmitting data, but it does not scale well. So for almost any structured data developers are forced to create and maintain databases, put query logic in the database and open and close database connections to read/write the data.

I recently heard Matt Mullenweg founder of WordPress.com discussing how WordPress uses MySQL simply as a dumb data store, no stored procedures, triggers etc. WordPress runs pretty much all the largest blogs and scales very well.

Clearly then, there’s a need for something a lot simpler than a full database – enter Tables, Azure’s answer to Amazon’s SimpleDB. In Tables data is stored in rows and columns but less structure is imposed than in database tables. Rules of normalization are relaxed and mixing different forms of data in a single table is now efficient. Tables don’t require connections like databases and more importantly they are blazingly fast and quick to develop.

Table storage goes some way to alleviate the biggest pain point for Azure adoption by smaller web developers – the minimum $10 per database which quickly racks up the monthly cost when several sites are added. Tables might prove a viable database alternative.

It will be very interesting to see how Tables evolve and which applications move over from  SQL Server and SQL Azure.