Storage Optimization


Posted in Featured by storageoptimization on April 7, 2008

Helping Address a New Set of Storage Challenges

In recent years, a handful of storage companies have built sizable and successful businesses by driving innovation in data reduction for areas such as back-up data and WAN data movement. Data Domain pioneered de-duplication, which has become the de facto standard for reducing the amount of disk space needed for backups.

However, the biggest and most expensive part of storage—the one that threatens to swamp Internet data centers with its upwardly spiraling load—is online data. Trying to fix this storage problem with these methods is like using a screwdriver on a nail. They’re the wrong tools for the job.

Online Data: A Different Animal

Whether you call them primary storage, online, or nearline, online data sets have different characteristics than backups. This translates to different needs for reducing their size. Backups are repetitive – you do them every day. If you’re backing up the same files over and over, it makes sense that there is going to be duplicate information that can eliminated. Several companies have dedupe solutions that do this, and there are pros and cons to each company’s solution. But the bottom line is that dedupe just doesn’t cut it when it comes to online data.

In your online set of files, there just isn’t a whole lot of duplication. Where there is duplicate information, it is often encoded differently from one file to the next. If I take a photograph and store it as a JPEG, then crop it and paste it in to a PowerPoint, and then scale it down and paste it in to a PDF document, it’s the same photo in all three places – but there might not be a single duplicate block of data that’s common across any of those three files. That’s because they are all compressed and encoded differently by the different applications that create and save those files. Even the best dedupe solution might not get much data reduction when presented with those types of files.

Dedupe Sample

Closing The “Dedupe Gap”

Deduplication may provide some benefit for online file sets, but it’s not really well-suited for addressing the growing needs of businesses that have enormous amounts of online file data. I’ll call that the “dedupe gap.” You want the benefit of dedupe, but to really deal with the massive growth of storage – most of which is in the form of online or nearline files – you need more. You need to be able to find and eliminate the redundant information in files at the information level, not the block or byte level.

A new generation storage optimization solution will have to address three big questions:

  1. Can you find and eliminate redundant information across a set of files, even when there are no duplicate blocks or strings of bytes at the disk level in those files?
  2. Can you provide data reduction for online files that have already been compressed using one or more generic compression tools?
  3. Can you provide data reduction where it is most needed – on storage that customers already have – without having to buy or implement a whole new tier of storage.

Companies that can solve all three will be able to close the dedupe gap and bring serious data reduction to the online and nearline space, delivering the same kind of benefits for online storage that dedupe has for backups.

Enter Ocarina Networks

Ocarina Networks Logo

Ocarina is a company that was founded to solve these problems. Ocarina’s three-step ECOsystem process – Extract, Correlate, and Optimize – is a new method for data reduction that can give you up to 10:1 data reduction for online storage, on your existing storage, from your existing vendors, without changing your storage management or backup processes.

Now that we’ve introduced ourselves, a bit about our the Storage Optimization blog. Here we’ll cover a wide cross section of topics having to do with data reduction – compression, dedupe, single-instancing, and where the research is headed – and we’ll welcome comments from everyone who has an interest in making online storage more efficient.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: