For new developers interested in audio, the world of compression algorithms, codecs and content formats is somewhat confusing. For this reason, I've decided to create an article that will help frame how it all works and comes together.

For starters, let's go over the differences between story, data, and data format.

Getting Started

Data vs. Story

Within industries such as news, music and TV, stories are usually called content but in the software world, they're more abstractly called data.

The nuance is subtle but one reason why I prefer to use the word "data" instead of "story" is because it removes all the emotion and ego from the story teller and the story itself. Said differently, it allows app producers and engineers talk about media products, services and platforms without having to focus on the actual content itself. In no way is an engineer trying to be dismissive, it's just easier to not have to focus on that aspect of the experience.

Data vs. Data Format

By switching story or content into data, then we can identify a slightly tangential topic. Data format.

In the music world, data is a song. In the film world, data is a film. In the gaming world, data is a game. Data can be both the story itself as well as the data format. So for example, a screenplay is content but the data format might be Microsoft Word. A song is content but the formats might be a Pro Tools project, an MP3 or even a CD. A film can be a Final Cut Pro project or a Blue Ray. In the web world, the data formats are usually:

• PNG / JPG for images
• JSON for text
• MP3/MP4/OGG for audio
• MP4 for video
• IPA for iPhone software or APK for Android software

Why is this important?

The content container is the format in which your media is stored. And if you want your story to reach the widest audience possible, then you'll need to think about the format your using and why you're using it.

In audio alone, we interact with different formats all the time including: .mov, .flac, .wav, .aiff, .SDII, .mp3, .mp4 etc. These containers each have different benefits but generally speaking, most people only care about these three things:

• License: Is this container open source or will I have to pay a license?
  • No License Required: wav, flac, mp3 (Hooray!)
  • License Required: mpeg, mp4
• User Adoption: If we want our data to be consumed by as many people as possible, then we've got to make sure that the format is accessible by PC users, Mac users, Linux, iOS, Android, Nokia, etc.
• Lossless / Lossy Comprssion: Is this format known for using lossless compression or lossy compression? This matters to some purists who want to listen to all music in high-definition. If your business caters to those users, then yes, you'll need to understand the subtle differences.
  • Lossy: mp3, mp4
  • Lossless: wav, flac, aiff, SDII

How do Codecs fit into this mix?

An encoder is the tool that compresses your audio or video into something small. A codec is essentially an algorithm inside your encoder that does all the work. If you have no idea what I'm talking about, start watching Season 1 of Silicon Valley and you'll get the gist of it.