The Daily Travails of a Reluctant DBA

MDD vs. TDD in Development

carpdeus@gmail.com — Fri, 12 Feb 2010 19:45:41 GMT

There has been a surge in recent years involving Test Driven Development, something that Mike Amundsen and I were discussing last night. It's not a development style that we work with much and that discussion led to this post and a new style of programming we call MDD, матрёшка doll development. матрёшка (Matryoshka) dolls are nesting dolls, looking something like this:

To understand the MDD, let's look at TDD and some of the reasons it seems to have exploded in popularity. One reason that I posited was that TDD is really great for quickly creating demo software. Create the stub of a class, wire it up to a UI and you have a working, though non-functional demo that can become functional as stub methods are replaced with functional code. Stubbing code is a good way to write code, but TDD requires such things, and requires writing the tests before writing functional code. But while this is a benefit to TDD, it turns out there is a very large downside to TDD and that is the tight coupling of code.

TDD is a way to force code to be specific and stateful. The code has to match consistently in all ways in order for it to work properly. And, while you may be saying to yourself that such tight binding is good, I suspect that the scenario can be broken down quickly to show that tight binding is not as beneficial as some folks want you to believe.

For an analogy, let's look at a house, built in TDD mode. It's a simple, shotgun house with a Living Room, Dining Room, Kitchen and Bedroom in that order. All we want to do is to change the dining room table from a small 2 person table to a larger 6 place setting. So we simply rebuild the entire shotgun house with the new dining room and all is fine and dandy.

Except that anyone who wants to enter the house now needs to know it has a dining room table for 6 instead of 2. Doesn't matter if they are doing nothing more than opening the front door to check to see if there's a package waiting to be delivered in the living room and have no knowledge of the Dining Room nor care about the Dining Room, they need to know that the Dining Room now has a bigger table because the dining room is tightly coupled to the living room.

Yes, a whole suite of tests can be run against that shotgun house to make sure nothing broke, but those test are probably going to also know the dining room table expanded, even if they don't go into the living room. Because tightly bound TDD is like the матрёшка dolls in that everything has to fit together exactly so and everything is statefully bound. And that, to me, is a major problem.

There is an answer, but it's not one that most people are going to like, and that's Stateless programming. Instead of tightly binding everything together, make sure every piece of code stands alone and doesn't require knowledge of anything else to get the job done. Let me open the living room door and see if there's a pacakge to deliver without requiring any knowledge of the rest of the house because I don't care about the rest of the house. The state of a package in the dining room doesn't matter to me, I only care about the package when it's in the living room.

When I think of Stateless programming, I think of 3 descriptions that give it an advantage over TDD:

It's more flexible. I can change up my dining room and it may or may not affect folks using the dining room but I certainly don't need to affect anyone who is using some other functionality.
That flexibility makes it more Agile. I can implement my changes without it affecting anything else.
It's more demanding to code statelessly. I'm not gonna lie about this, but the extra demands in writing stateless code provide the flexibility that's missing from TDD. And that's an advantage because it's that minor inconvenience of writing more demanding code that creates the flexibility and agility of stateless code.

I'm sure that there are a lot of programmers and architects who have read that list and are ready to start telling me that I am mis-using the word Agile and that TDD provides very Agile paradigms. But they are using Agile in a specific discipline. And TDD is agile like a Ford Mondeo while stateless programming is agile like a Ferarri 599. To understand the difference, let's take a quick look at the dreaded F word that Microsoft is so fond of, Frameworks.

I think everything coming out of Redmond lately has a three letter acronym (TLA) ending in F. And there are times when a Framework makes a lot of sense, but not all the time. Using a Framework in your application can be very much like installing a full-blown transportation infrastructure, complete with subways and buses when all you want to build is a dirt track for you go-cart. Even if you're plan is to build a stock car track, you're not going to be using subways or buses or ferries. But a lot of projects use Frameworks and I think I know why.

Microsoft likes to say they have taken care of the plumbing that we programmers can focus on more challenging issues. Over the years, however, that plumbing has grown in complexity to the point that using the various Frameworks is less helpful than not; rather, it is more helpful to start but far less helpful as you go along.

Let's say that you're on a team that's starting out a new project. Using the various TLA's Microsoft has provided, you can quickly put together a prototype, kind of like going to a large rental company and asking for transportation and getting a fleet of Ford Mondeos. They are perfectly adequate cars for transportation, easy to get and everyone knows how to use them. But will they actually take your project where it needs to go?

There are companies in Minnesota that retrofit your cars to be more prepared for winter by replacing tires with tracks, something like this:

Retrofitting a vehicle can be far more expensive than designing it the way you need from the beginning. It can also be more time-consuming. And when managers see a demo whipped up in a couple of weeks they wonder why it takes so long to get a real project out the door and finished. This is one consequence of using a framework incorrectly and an expense in both dollars and time that applies to many projects.

How can you tell when are you using a framework incorrectly? When the framework constrains your programming, forcing you to contort your program into the shape of the framework:

A man goes into a tailor shop to buy a new suit. The tailor helps him try one on. It looks good. But then he sees that one leg is too long. The tailor suggests that he hitch it up. Then the arm is too long. He hitches that up, and so on. The man is confused but everyone has told him that this is the best tailor in town so he accepts the adjustments, he admires himself in the glass and purchases the suit. As he hobbles down the street holding his suit carefully in place, two men come along. One exclaims, “Look at that poor man!” The other replies, “Yes, but he's got a mighty fine tailor.”

When you start hobbling your program to fit the constraints of the framework, you're doing it wrong. If your framework makes you build out an entire transportation system to enable you to race cars around a track, you're doing it wrong. And if your framework requires everyone to know the size of the dining room table to check for a package in the living room, you're doing it wrong.

Finally, if your framework has tight binding and requires state knowledge outside of the piece you're working on, you're doing it wrong. All state should be self-contained. If it's not, you're increasing your scalability issues and hobbling your program.

I'll be talking a great deal more about state and non-framework programming over the next several weeks as I finally launch a new series on using Azure for building truly scalable projects.

And I'll be doing my best to avoid матрёшка doll development.

Josef Finsel

Azure-Architect.com

Technorati Tags: Azure,Test Driven Development,TDD,MDD,матрёшка Doll Development,Stateless vs Stateful,REST

Azure SQL Scripter

carpdeus@gmail.com — Tue, 10 Nov 2009 10:00:17 GMT

So, you've got an Azure SQL Database and have been doing some work in it/on it and then find out it is a CTP and you need to move the data and schema to another database.

Which isn't bad but there aren't any good tools for scripting an Azure SQL database, mostly because Azure SQL doesn't support using the DMO, which is what most people use.

So, I wrote my own.

It's not perfect, it's missing a few things (doesn't handle triggers, for instance) but it should be good enough to get you started.

It's simple enough. Download and run the code (or Install the app), enter the connection string to the server and a file name to store the script and click on Backup. The utility will walk the database, scripting tables, constraints, indices, views, stored procedures and functions.

Each object is wrapped in an IF EXISTS so you should be able to run it multiple times.

If there's something missing and you want to add it, add it and send me the code and I'll incorporate it.

If there's something missing and you don't know how to add it, leave feedback and I'll see if I can incorporate it.

Keep watching both this space and CloudQuotes. I've got some exciting new projects that will be pushing some boundaries, including a handler for AzureStorage apps.

Josef Finsel
AzureDBA

Technorati Tags: SQL Azure,Utilities

10,000 Lines of Code in 10 Minutes!

carpdeus@gmail.com — Thu, 13 Aug 2009 12:51:07 GMT

The title of this post comes from an ad for one of those code generators that scans your database and creates an application with little to no work on the part of the developer. Sounds impressive, but what would impress more is the right 10,000 lines of code. Mark Twain said it best:

The difference between the almost right word & the right word is really a large matter--it's the difference between the lightning bug and the lightning.
- Letter to George Bainton, 10/15/1888

The same goes for code. If you want a simple example, look at sorting algorithms. Bubble sort is the one that almost everyone comes across when first learning to sort data, and they quickly find that there are other ways to do it, some of which are more efficient in certain situations than others. The difference between the right sort routine and the wrong sort routine is the difference between a scalable program and one that has limitations.

Within limitations, I don't care if the code is 10,000 lines or 10 lines, I care that it does the job I want as efficiently and flexibly as possible. And it's that last part that causes the most consternation when using code generators. Creating a DataSet class in Visual Studio works fine for rough prototyping but I find it's almost as quick and easy to write the class from scratch so it does what I want it to do, not just come close the way a code generator does.

Just my .02

On Starting Over

carpdeus@gmail.com — Fri, 24 Jul 2009 12:39:58 GMT

Luciano E. Guerche was recently lamenting the costs of changing frameworks, especially within the Microsoft world. He said:

If you want Winforms app to turn a WPF app? Rewrite it from scratch? Come on M$...

To which I replied:

(re)Writing from scratch enables you to avoid the bugs in the original app and write brand new ones! (yes, bugs)

It was tongue in cheek and very reminiscent of an old saw about bugs in code:

Every program has at least one bug and can be shortened by at least one instruction from which, by induction, one can deduce that every program can be reduced to one instruction which doesn't work.

But it also had a couple of truths in it. Frameworks are wonderful things. Engineers and architects use frameworks all the time. The Statue of Liberty is copper over an internal framework of steel. But changing that framework isn't easy. And the same thing applies to writing code.

Microsoft has a litany of Frameworks, three letter acronyms that end in F (WCF, WPF, etc) and all designed to help programmers "not have to be plumbers". And I'm all for not having to re-do the internal plumbing every time, but that doesn't mean I need a framework. Frameworks are like jigsaw puzzle pieces and they only tend to go together one way and they build the exact same thing each time. What I, as a programmer, need is a set of tools like Legos or Tinker Toys or Lincoln Logs. And the beauty of all three of those items just mentioned is I can and have used them together to make some incredible things.

Not only that, if a part of it doesn't work for what I need any more, I can dismantle that one part and change it without having to rebuild the entire structure, which is what Luciano was complaining about.

Just my .02 for today

Pool Lessons

carpdeus@gmail.com — Fri, 10 Jul 2009 13:03:05 GMT

I used to work in an office that had a pool table, one of those really expensive, championship caliber ones. Since it didn't cost me anything to use it, I would use it a lot as a break from my net time. And I come to some conclusions that I wrote back then. Not quite a "Everything I need to know I learned from pool" but some things that are as applicable now as they were 7 years ago....

Finesse can be more important than power.
Oh how true. Like many things in life, I started out shooting pool by just trying to ram things through. I can be quite powerful and when I break with all my might, the balls scatter to the four corners of the table. But most of the time finesse is required over brute force. If you have the perfect shot lined up, particularly a straight shot, and power the ball into the pocket, odds are the cue ball will follow it in and you've wasted that shot.
Remember the goal of the game.
When I was in college, back in the mid-80s, I got hooked on 9-ball. The rules are simple. You have to hit the lowest numbered ball on the table. The object of the game is simple. Sink the 9-ball. As I was playing this weekend I had a nice shot lined up and hit the 4 which banked off and sank the 9, effectively ending the game. Since then, I've paid more attention to where the 9 is and whether it can be pocketed. Before that, my sole focus was on sinking the lowest numbered ball. The same applies to life in general. Getting caught up in the day to day minutuae can lead us to losing sight of our ultimate goal.
Hail Mary's may work but don't count on them.
This is an old sports term. You know, end of the game, your just a little behind and you throw a pass the length of the field or attempt to sink a basket from the other end of the court with little more than a prayer that it will work. I find myself doing that sometimes when shooting pool. Sometimes because I'm lazy or because I don't have a good shot I just wind up the cue and let loose. Earlier today I sank three balls on one shot that way. Mighty impressive sounding but I couldn't duplicate it so it really isn't all that great. I mean, if I did that on purpose when it counted, that would be one thing. But the shot was totally unplanned and a result of pure luck.
Know the difference between sinking a ball and sinking the right ball.
Sometimes I'll shoot straight pool and actually alternate between stripes and solids as though I were two players. And sometimes I just rack the balls and shoot whatever is available. The key is that you have to remember whether you are supposed to sink a specific ball or if you can sink any ball. It's that way with the decisions we make in life too.
Don't forget to look ahead.
Pool, like chess or any other game, requires more than just the shot you are attempting to make. It also requires that you think about where you want to leave the cue ball so you can make the next shot. And that requires practice, just like in real life.

A Parable Applied to Developers

carpdeus@gmail.com — Sat, 04 Jul 2009 19:55:35 GMT

One of my friends was recently musing on whether they should attend DevLink or not and, in their musings, said it all depended upon what the SQL Tracks were going to be. For some reason, that particular incident reminded me of the parable of the Beam:

Judge not, that ye be not judged.
For with what judgment ye judge, ye shall be judged: and with what measure ye mete, it shall be measured to you again.
And why beholdest thou the mote that is in thy brother's eye, but considerest not the beam that is in thine own eye?
Or how wilt thou say to thy brother, Let me pull out the mote out of thine eye; and, behold, a beam is in thine own eye?
Thou hypocrite, first cast out the beam out of thine own eye; and then shalt thou see clearly to cast out the mote out of thy brother's eye.
Matthew 7:1-5

Regardless of the original intent of the parable, I think it's very applicable to developers (myself included) when it comes to technology in general and programming languages in specific.

I would say that most developers who have been around for more than a couple of years have been forced to learn a new technology, even if it's only the new version of their current technology. And most of us are very good at some part of that technology. I can list a lot of developers who really rock at SQL Server or C# or VB(.NET) or, well, you get the idea. But that very focus can have a detrimental effect on us as developers if we reach a point where we filter everything through that technology, an effect that seems to manifest itself in two different ways.

The first is the "My technology is the greatest thing in the world and it's a true panacea." SQL Server is great, but I really wouldn't want to write a T-SQL stored procedure to handle a large amount of string processing, because that's something SQL Server does neither well nor efficiently. And yet I have known a developer or two whose answer to everything is, "We can do that in the database." Not many but still...

The second, and far more insidious effect is that of putting blinders on that filter out anything not related to our chosen technology. When .NET first came out and C# was introduced, there were a lot of VB programmers that took the "They can have my VB when they pry it out of my cold, dead hands" stance, some of whom even refused to look at the code. The real shame of putting on blinders that limit our technology focus to our chosen technology is that it artificially eliminates items that may not be totally relevant to our job but have an impact on how we do our work.

There have been times that attending sessions or reading about a technology totally unrelated to mine has made me rethink some of the ways I'm handling things in my day to day technology. Sometimes those have been directly beneficial. Sometimes they have not worked but led me down a path that was. And sometimes it ended up being just an interesting intellectual exercise following along. But in each of those cases, I was sharpening my saw, which is always a good thing.

Technorati Tags: Parable,Saw Sharpening,Blinders

Grouping in a Sort

carpdeus@gmail.com — Wed, 01 Jul 2009 19:21:42 GMT

RowNumber() really is a great addition to SQL Server and has been very helpful in many of my applications. But I have been granted a new task, adding grouping within a sort. To explain the dilemma, let's look at some rows of data that are sorted in date order:

RecID GroupTag SortColumn RowNumber

1 NULL 2009/2/13    1

2 NULL 2009/2/13    2

3 GroupSet 2 2009/2/14    3

4 NULL 2009/2/14    4

5 GroupSet 2 2009/3/01    5

6 GroupSet 2 2009/3/10    6

7 GroupSet 2 2009/3/10    7

RecID	GroupTag	SortColumn	RowNumber
1	NULL	2009/2/13	1
2	NULL	2009/2/13	2
3	GroupSet 2	2009/2/14	3
4	NULL	2009/2/14	4
5	GroupSet 2	2009/3/01	5
6	GroupSet 2	2009/3/10	6
7	GroupSet 2	2009/3/10	7

What I need to do is sort it so that RecIDs 5-7 show up between 3 and 4, because they belong to a group (defined by the GroupTag) and should be grouped together with the first record in the sort of that group.

To make things slightly easier, I'm actually selecting a subset of data and using the RowNumber() function to put it in order and support paging. The trick is to resort to adding a column called GroupSort to the data to apply grouping within either a temporary table or table variable. Now for the tricky part (or at least the part I'd like to refactor away). If the GroupTag is null, I set the value of the GroupSort column equal to the RowNumber column. If GroupTag isn't, then I do an update that fills GroupSort with the minimum RowNumber for the GroupTag plus the current RowNumber/1000. This gives me the results seen below:

RecID GroupTag SortColumn RowNumber GroupSort

1 NULL 2009/2/13    1    1

2 NULL 2009/2/13    2    2

3 GroupSet 2 2009/2/14    3    3.003

4 NULL 2009/2/14    4    4

5 GroupSet 2 2009/3/01    5    3.005

6 GroupSet 2 2009/3/10    6    3.006

7 GroupSet 2 2009/3/10    7    3.007

RecID	GroupTag	SortColumn	RowNumber	GroupSort
1	NULL	2009/2/13	1	1
2	NULL	2009/2/13	2	2
3	GroupSet 2	2009/2/14	3	3.003
4	NULL	2009/2/14	4	4
5	GroupSet 2	2009/3/01	5	3.005
6	GroupSet 2	2009/3/10	6	3.006
7	GroupSet 2	2009/3/10	7	3.007

When I finally output this data, it will correctly sort the RecIDs the way I want.

Inelegant, but it works. Share a better solution in the comments.

AzureDBA.com is Open

carpdeus@gmail.com — Mon, 29 Jun 2009 15:30:12 GMT

I know I've put a lot of Azure information in this blog and more people expect it to be geared more toward SQL Server and C#. So I've created a new website, AzureDBA.com, to host all of my Cloud Computing information; including, but not limited to:

Best Practices
Codes Samples
Forums for discussions

Stop by and join the discussion of what will be an important part of the next step in computer programming.

Josef

A REST-ful Look at Azure (Part 9) Deeper Dive on Worker Roles

carpdeus@gmail.com — Thu, 18 Jun 2009 12:27:58 GMT

In my last post, A REST-ful Look at Azure (Part 8) Azure Hosted Services, I took a very brief look at the Web Role and a more in-depth look at the Worker Role. Today I'm going to explore some things you should do when implementing a Worker Role.

As I said in my last post, the Worker Role should be an extremely light wrapper that calls a class that does all of the real work. There are two good reasons for this. First, the less code in the Worker Role, the less debugging you need to do in the Development Fabric, the easier it will be to write, test and debug your code. Second, the code in the Worker Role should be fairly stable. If all it is doing is instantiating a class and calling it every few seconds, it lessens the chances of errors happening in the Cloud version that will require major changes. Follow the basic rule of Keep It Super Simple when it comes to your Worker Role.

The class that the Worker Role is instantiating should include a call back function (see my blog post Services, Worker Classes and Templates, Oh My from February 23, 2009 for more details) that serves two purposes. First, it allows the Worker Role to log information to the Cloud Logs but it also updates the DateTime value the Worker Role uses to determine it's not hung. Finally, it should also include a way for the Worker Role to notify it that a request has been made to stop processing.

My generic Processing Class simply queries Azure Queue Storage to see if there is a message to be processed. There may be more than one, but the Processing Class gets just one and processes it. When it has finished, it checks to see whether or not the boolean value stopProcessing is true. If it isn't, then it checks the queue for another message and starts the process all over again.

While this is the best solution we have at the moment, it's not the optimal one and will have to change as Azure Hosted Services comes out of the CTP stage and enters Beta or Release. A couple of items this doesn't address:

How responsive must a Worker Role be?
If a Windows Service doesn't respond to a shutdown request in 30 seconds, it's considered "frozen" and Windows will instantly terminate the thread. It's unknown what the Worker Role will do or how long it will wait. It will probably be a requirement to handle a graceful shutdown that every time you call Notification you also check to see if you should stop processing.
Continuation of processing.
If the Worker Role ends in the middle of a request, how do you gracefully continue? How Idempotent is your processing? In this particular example, if the request isn't in Pending status, the worker role just says it is in error and doesn't do anything.

Code Samples

The complete sample code can be downloaded here.

First, we have the basic plumbing that handles sending notifications back to whatever is calling the class:

  enum WorkerRoleEventLogType
  {
   Critical,
   Error,
   Warning,
   Information,
   Verbose
  }

  /// 
  /// Get messages from the Class as it processes
  /// 
  /// Message data
  public delegate void LogHandler(string messageType, string message);

  private LogHandler mvLogHandler;

  public void OnLogHandler(LogHandler msgString)
  {
   if (msgString != null)
   {
    mvLogHandler = msgString;
    Console.WriteLine(msgString);
   }
  }

  /// 
  /// Notify the calling object of an event
  /// 
  /// Message of what has happened
  void NotifyCaller(WorkerRoleEventLogType logType, string msg)
  {
   if (mvLogHandler != null)
    mvLogHandler(logType.ToString(), msg);
  }

Since we are using our Worker Class to handle all logging, we have enum that allows us to tell the Worker Class what type of logging we are doing along with the basic NotifyCaller method that will either have a handler defined for it or not. If the calling object has defined a handler for NotifyCaller, then we call it, passing in the parameters. If it hasn't then we just ignore the request made to NotifyCaller.

We have at least one property that we need to be able to set:

public bool stopProcessing { get; set; }

This property is used to to let us know that we need to stop processing because the Worker Role is shutting down. Just like with a Windows Service, our class needs to be able to wrap up whatever it's doing and stop processing if necessary. This value is set to True in the AcceptShutdown() and Stop() methods of the Worker Role.

Next we have the Initializer for the class. This takes the basic information needed for whatever job this DLL needs to do. In this case, it will be getting queue information and processing bulk Azure Table Storage requests so the Initializer contains authentication information:

  public bulkProcessing( string Account, string Key, string Tag)
  {
  dfltAccount = Account;
  dfltKey =Key;
  bulkTag = Tag;
  }

Note: Any values for the Initializer should be defined in the Service Configuration so that you can change them without the need to upload a new version of the software.

All of my classes that are designed to be called from Worker Roles or Windows Services all have a Process method which handles all of the work:

 public void Process()
 {
  AzureQueueStorage aqs = new AzureQueueStorage(dfltAccount, string.Format("http://{0}.queue.core.windows.net", dfltAccount), dfltKey, "SharedKey");
  azureResults ar = aqs.Messages(cmdType.get, bulkTag, "", "visibilitytimeout=7200", "");
  string MessageID = string.Empty;
  string PopReceipt = string.Empty;
  string Message = string.Empty;
  string summaryKey = "_Summary";
  while (ar.Body != null)
  {
   // Do processing code here
   if (!stopProcessing)
    ar = aqs.Messages(cmdType.get, bulkTag, "", "visibilitytimeout=7200", "");
   else 
    break;
  }
}

This code is getting a message off of the queue, if one exists, and processing it. When it's finished processing, it checks to see if the stopProcessing property is true. If it isn't, it looks for another message. If it is, it stops processing.

That's the simple method of writing a class to be instantiated by a Worker Role. If you have more complex processing requirements or your processing may take longer, you may need to use threads, a concept that I'll tackle sometime soon, when I've finished the basics.

That's all for today. In the next installment we will look at handling transactions in Azure Table Storage.

Technorati Tags: Azure Hosted Services,Cloud Computing,Azure Worker Roles,Best Practices

All Posts in Series:

Why Everything You Know as a DBA Loses Validity in the Clouds

carpdeus@gmail.com — Wed, 17 Jun 2009 13:36:18 GMT

You may disagree with the title of the post, but I think it is far more true than not. First, because I know folks are going to try to tell me, this doesn't apply if you're using S[S]DS (SQL ~~Server~~ Data Services), once it finally becomes available because it really is nothing more than SQL Server running on MS machines in the cloud. This is, of course, the same as if you were running SQL Server on Amazon hardware. But there's a major problem with SQL Server when it comes to scalability above a certain level, most of which I covered in Why Aaron Skonnard is Wrong About Azure Table Storage. And all of the things that make SQL Server bad for highly scalable web sites are the things we know as DBAs.

For instance, if you've ever heard Kimberly Tripp speak about database design, you've probably heard her mantra about clustered indices being "monotonically increasing values" (Identity Columns) because SQL Server is built to handle those especially well as clustered indices. That's all well and good but Azure Table Storage, while it may be using SQL Server as a back end processor, doesn't care about Identity columns and ATS certainly doesn't support them. If you want to create Identity columns you'll need to roll your own. But I wouldn't recommend it because it creates a choke point, and creating highly scalable data means avoiding those.

Take all of the best practices related to building indices for SQL Server to support JOINs. A great feature of the current SQL Server is the ability to create covered indices without including the covered columns in the actual index. ATS doesn't support JOINs so covered indices are out.

Any of the many tips and tricks we've learned over the years in working with SQL Server related to writing stored procedures are out because ATS doesn't support stored procedures.

Take a look at Microsoft's SQL Server Best Practices page. Almost everything there is very SQL Server specific and wouldn't apply to an Oracle database or an Informix database. Which is why none of these will apply to ATS, because ATS is not SQL Server. Not only that, many of the rules of Normalization do not apply to ATS because Normalization applies to Relational data and ATS is not relational.

In a very Zen-like way, we need to discard almost everything we take for granted as knowledge about how to work with data and re-focus on the data itself, how we can work with it and what we need to do with it. Not only that, Azure DBAs are going to need a much bigger chunk of .NET programming to be able to handle the tasks ahead of them.

Scalable data requires rethinking our expectations and a huge paradigm shift. I'm going to be working through a lot of the issues related to Azure Hosted Services and Azure Storage on my new site: AzureDBA. It'll be going live in the next week with most of the thoughts from the Azure postings I've done here along with sections on best practices for all aspects of scalable cloud computing and any new thoughts I come across as I continue to work with and explore Cloud Computing with Azure.