Microsoft Azure

Why Everything You Know as a DBA Loses Validity in the Clouds

carpdeus@gmail.com — Wed, 17 Jun 2009 13:36:18 GMT

You may disagree with the title of the post, but I think it is far more true than not. First, because I know folks are going to try to tell me, this doesn't apply if you're using S[S]DS (SQL ~~Server~~ Data Services), once it finally becomes available because it really is nothing more than SQL Server running on MS machines in the cloud. This is, of course, the same as if you were running SQL Server on Amazon hardware. But there's a major problem with SQL Server when it comes to scalability above a certain level, most of which I covered in Why Aaron Skonnard is Wrong About Azure Table Storage. And all of the things that make SQL Server bad for highly scalable web sites are the things we know as DBAs.

For instance, if you've ever heard Kimberly Tripp speak about database design, you've probably heard her mantra about clustered indices being "monotonically increasing values" (Identity Columns) because SQL Server is built to handle those especially well as clustered indices. That's all well and good but Azure Table Storage, while it may be using SQL Server as a back end processor, doesn't care about Identity columns and ATS certainly doesn't support them. If you want to create Identity columns you'll need to roll your own. But I wouldn't recommend it because it creates a choke point, and creating highly scalable data means avoiding those.

Take all of the best practices related to building indices for SQL Server to support JOINs. A great feature of the current SQL Server is the ability to create covered indices without including the covered columns in the actual index. ATS doesn't support JOINs so covered indices are out.

Any of the many tips and tricks we've learned over the years in working with SQL Server related to writing stored procedures are out because ATS doesn't support stored procedures.

Take a look at Microsoft's SQL Server Best Practices page. Almost everything there is very SQL Server specific and wouldn't apply to an Oracle database or an Informix database. Which is why none of these will apply to ATS, because ATS is not SQL Server. Not only that, many of the rules of Normalization do not apply to ATS because Normalization applies to Relational data and ATS is not relational.

In a very Zen-like way, we need to discard almost everything we take for granted as knowledge about how to work with data and re-focus on the data itself, how we can work with it and what we need to do with it. Not only that, Azure DBAs are going to need a much bigger chunk of .NET programming to be able to handle the tasks ahead of them.

Scalable data requires rethinking our expectations and a huge paradigm shift. I'm going to be working through a lot of the issues related to Azure Hosted Services and Azure Storage on my new site: AzureDBA. It'll be going live in the next week with most of the thoughts from the Azure postings I've done here along with sections on best practices for all aspects of scalable cloud computing and any new thoughts I come across as I continue to work with and explore Cloud Computing with Azure.

A REST-ful Look at Azure (Part 8) Azure Hosted Services

carpdeus@gmail.com — Thu, 11 Jun 2009 12:42:29 GMT

Windows Azure isn't just about Azure Storage Services, it's also about running hosted programs in the sky. And this is actually a critical part of Azure Storage Services, even though it may not seem like it. The series so far has introduced Azure Storage Services and talked about what you can do with each of the pieces, but the time has come to take it to the next level, which includes taking care of some of the management issues that SQL Server and other RDBMSs do for us.

Now, I have said that one of the reasons that Azure Table Storage scales better than SQL Server is because it is simpler and doesn't have all the overhead of managing the data that SQL Server does. And you might accuse me of reinventing the wheel. But there are certain pieces of data management that make sense. Take what SQL Server does when we ask it to process something like this:

More...

Why Aaron Skonnard is Wrong About Azure Table Storage

carpdeus@gmail.com — Wed, 10 Jun 2009 23:28:51 GMT

Update: Aaron responded to my post (please see comments below) so I've made a minor edit to the post.

At this morning's first Azure talk at VSLive Las Vegas, I thought I heard Aaron Skonnard state that Azure Table Storage was irrelevant, to be replaced by SQL Data Services (his views are in the comments) . I have to say, that probably didn't really help with attendance at my after lunch talk on ATS, but it did get me thinking even more about why ATS is important and why SQL Server has scalability issues.

Let me state, first and for the record,

More...

Is it so much to ask, Microsoft?

carpdeus@gmail.com — Tue, 09 Jun 2009 21:01:13 GMT

So, I'm testing something with Azure Table Storage and notice that the data I get back is not the data I thought I sent. So I stepped through my class carefully. What I sent to MS to store in my table was:

More...

A REST-ful Look at Azure (Part 7) Blob Storage continued

carpdeus@gmail.com — Tue, 09 Jun 2009 15:18:32 GMT

Creating and Retrieving Blobs with Blocks

If you want to upload a large file to Azure Blob Storage (ABS), you're going to need to split it into pieces and transmit the individual pieces. Then, when you get them all uploaded, you tell the server to assemble them. This is all done using Blocks, which may be up to 4MB in size. It sounds complicated, but it isn't really. The first thing you do is split the file into 4MB blocks. If you have a Byte Array, this is easy enough to do. Then, you upload each block to the server using PUT and the URI of http://{account}.blob.corewindows.net/{Container}/{Blob}?comp=block&blockid={blockid} (CanonicalURL is

More...

A REST-ful Look at Azure (Part 6) Blob Storage

carpdeus@gmail.com — Mon, 08 Jun 2009 21:27:04 GMT

So far, this series has covered an Introduction to Microsoft Azure, the way that Azure Storage handles Security, an Introduction to Azure Table Storage, more on Azure Table Storage and Azure Queue Storage. Today I venture into Azure Blob Storage. If you want to follow along, download a copy of my AzureCommand class. You also might want to create an Microsoft Azure Account. I should state that I am not looking at the locally hosted development storage, only at the cloud hosted one.

When we talk about BLOBs, we are talking about Binary Large Objects, not an amoeba like alien that terrorized Downington, PA (although it appears that a blob is a blob and not really a BLOb). And BLOBs generally can be thought of as files. These files exist in...

Containers

Containers in Azure Blob Storage (ABS)

More...

A REST-ful Look at Azure (Part 4) Table Storage (cont.)

carpdeus@gmail.com — Thu, 04 Jun 2009 14:14:40 GMT

So far I've covered an Introduction to Microsoft Azure, the way that Azure Storage handles Security , an Introduction to Azure Table Storage, and Azure Queue Storage. Today I'm going to look deeper at some aspects of Azure Table storage that didn't make it into the introductory post. If you want to follow along, download a copy of my AzureCommand class. You also might want to create an Microsoft Azure Account and load in some data. If you're looking for some data to play with, load a copy of test data. I should state that I am not looking at the locally hosted development storage, only at the cloud hosted one.

eTags

HTTP Etags are

More...

A REST-ful Look at Azure (Part 5) Queue Storage

carpdeus@gmail.com — Wed, 03 Jun 2009 13:08:33 GMT

When one thinks Queues, one may conjure up many different images. Microsoft has MMQ and has implemented Service Broker, both as means of having stateless communication. Azure Queue Storage (AQS) is similar. The stated reason for using AQS is to be able to kick off Azure Worker Roles, but you can actually use AQS for any type of stateless communication. AQS consists of two parts: Queues and Messages. Queues, like Tables in ATS, are really nothing more than defined URIs that can be used to store and retrieve messages.Messages are nothing more than text packets (up to 8KB) that can be stored and retrieved from a specific URI. So, let's dig in.

Queues

Queues in AQS, are

More...

A REST-ful Look at Azure (Part 3) Table Storage

carpdeus@gmail.com — Tue, 02 Jun 2009 11:00:26 GMT

As Microsoft uses a REST-ful API, I'll be talking about what HTTP Method needs to be called; what special headers, if any, need to be included, and what you can expect to get back. If you have an account with data, you can follow along at my Azure Table Storage Web Interface, where you can enter your own account information and see what happens.

Also, before we dig too deeply, ATS uses an Atom format. If you can read XML, you should be able to follow the data. You may want to brush up on XML with Namespaces but the code will cover everything we need.

Azure Table Storage (ATS)

Azure Table Storage (ATS) is a slightly misnamed tool.

More...

A REST-ful Look at Azure (Part 2) Security

carpdeus@gmail.com — Mon, 01 Jun 2009 11:00:34 GMT

In last Friday's post, I briefly introduced Microsoft Azure and talked about Azure Storage. Today I'm going to continue talking about Azure Storage by talking about Security and how it's implemented. If you want to follow along, download a copy of my AzureCommand class.

There are many ways to implement security on the web. The original SQL Server Data Services that Microsoft debuted in 2008 used basic auth. For whatever reason, Microsoft decided that wasn't the way to go so they implemented two different security schemes for Azure, one for Table Storage and one for both Queues and Blobs. But they share some similarities so we'll talk about them both at a high level before we dig into the specifics.

More...