Today, creativity outsourced. The University of Houston presents this series about the machines that make our civilization run, and the people whose ingenuity created them.

In 2006, Netflix had both a solution and a problem. At issue was the question of video ratings. Based on a person's ratings of past videos, what would be her rating for, say, The Sound of Music or The Hitchhiker's Guide to the Galaxy? The better the predictions, the better the recommendations Netflix could make.

Netflix's description of "The Sound of Music" Photo Credit: Andy Boyd

The solution came in the form of Cinematch, an algorithm developed by a team at Netflix. The problem was that they weren't sure if there was a better algorithm. After all, there were limitless ways to predict rankings. Had the team chosen the best?

So Netflix took an innovative step. The company posted an online challenge asking the world if someone could better the Cinematch predictions by ten percent. A prize of one million dollars would go to the first team, if any, that could achieve this goal. The company posted historical data online and waited for responses.

Netflix had struck a chord. The idea of posting a challenge was so creative it quickly caught on, with other companies posting their own challenges. Soon, websites emerged for the sole purpose of hosting these challenges.

A visit to one of the more popular hosting sites gives an idea of what companies are looking for. A large auto insurance company wants to know if it's possible for computers to recognize distracted driving. Competitors are provided with a large database of pictures showing people behind the wheel and asked to pick out people who are texting, putting on make-up, or similar activities. Six-hundred and sixty-five teams are competing for a prize of $25,000.

State Farm tries to inform customers about distracted driving. Photo Credit: Andy Boyd

In another instance a well known travel website is looking to improve hotel recommendations. With hundreds of millions of visitors to the website each month, increasing sales by only a small percent can make a huge difference. Nine-hundred and forty-five teams are competing for a prize of $65,000.

Not every problem relates to business. What are the effects of lifestyle on aging? Will a blood donor return to give more blood in the future? What restaurants are most likely to need health inspections? If a problem lends itself to data analysis it's fair game. And in our computer dominated world there's a lot of data just waiting to be analyzed.

Many listings don't involve a cash prize but still attract a lengthy list of competitors. For individuals who are drawn to such challenges it's like playing a game. Winners get bragging rights and a nice addition to their resumes. And not surprisingly, companies use the competitions to recognize talent for recruiting.

As for Netflix? The competition attracted thousands of teams from 186 countries. It also supplied untold hours of free publicity. And after three years a winner was crowned. Actually, two teams tied, but by the rules of the competition the team with the earlier submission won. The two submissions arrived a mere twenty minutes apart. That's a heartbreaker.

The winners of Netflix's competition. Photo Credit: Andy Boyd

I'm Andy Boyd at the University of Houston, where we're interested in the way inventive minds work.

(Theme music)

For a related episode see LINEAR ALGEBRA AND NETFLIX.

Examples of crowdsourced data analytics problems were taken from the following hosting sites: https://www.kaggle.com/competitions, https://www.drivendata.org/competitions/, https://www.crowdanalytix.com/community. 

Netflix Prize. From the Wikipedia website: https://en.wikipedia.org/wiki/Netflix_Prize. Accessed May 10, 2016.

This episode was first aired on May 12, 2016