Algorithms: Turning Data Into Products - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Algorithms: Turning Data Into Products

Everybody talks about algorithms, but do you know what they are? We take a closer look at the origins of the word algorithm, what it means for business today, and how companies such as Google and Airbnb use them.

9 Scripting Languages You Need To Know
9 Scripting Languages You Need To Know
(Click image for larger view and slideshow.)

Blame Persian mathematician Al-Khwarizmi. Around 825 AD, he wrote a book entitled "Al-Khwarizmi on the Hindu Art of Reckoning," basically a math book using the numbers we use today. It was translated into Latin as "Algorithmi de numero Indorum." From this, we get our current word "algorithm".

Every software developer knows what an algorithm is -- a set of rules that list a set of instructions. The rest of us rely on algorithms for much of our daily Web and mobile interactions, though we're not always conscious of the important role they play.

In fact, at the Gartner CIO Symposium in October, the research firm's CEO Gene Hall proclaimed that we are living in "The age of the algorithm." While your ability to build effective algorithms depends upon the quality of your data, the data itself is effectively useless without algorithms that can tease out meaning. "Data is inherently dumb," said senior vice president of Gartner Research Peter Sondergaard at the October event. "It doesn't do anything unless you know how to use it and act with it. Algorithm is where the real value lies. Algorithms define action."

[ Find out why algorithms matter so much. Read CIOs Need To Welcome Age Of The Algorithm, Gartner Says.]

We recently spoke with developers from Google and Airbnb, two businesses whose fates rise and fall on the quality of their algorithms, to find out more about what algorithms are, and how they function as the underpinning of entire business models.

As with any development process, though, the action starts with defining your goals and what you're trying to achieve, said assistant professor Ariel Procaccia, in an interview. Procaccia teaches computer science at Carnegie Mellon University. "You start with the problem, not the algorithm."

Once you've identified the problem you're trying to solve -- or the business result you're trying to achieve -- the algorithm sets forth the steps that will get you where you want to go.

"An algorithm is an ordered set of instructions," said Kourtni Marshall, a software engineer for Google. "Writing down what happens after something happens should terminate in a satisfied condition." 

Algorithms rely on data. If you can get results from clear data, you can spot patterns, such as why a query is returning the wrong results, according to Surabhi Gupta, engineering manager at Airbnb.

For companies such as Google and Airbnb, algorithms are their "product." In the simplest terms, a Google search algorithm has to produce an effective search result. That search result is the company's product. Airbnb's algorithms have to connect travelers with those offering accomodations. Practical expediency governs how algorithms are used and referenced.

In the business world, algorithms are built for speed. "You are working with people with the expectation of immediate answers," said Google's Marshall. "Your algorithm has another constraint on it."

In the end, a developer is taking a sub-optimal solution that runs fast, and comparing it to the optimal solution to see how close the two compare, he added. If a problem is computationally simple, it can be solved quickly. If the problem is difficult, then the developer may have to compromise, preferring "good enough" instead of "perfect" in order to get the job done quickly enough, said Carnegie Mellon's Procaccia.

At Airbnb, the process of developing an algorithm involved a repeated build-test cycle, as the algorithm is improved to obtain a good result. Then the solution undergoes online A/B testing to see if there is any significant change between two possible solutions. Once all is cleared, the "product" is ready to go online, Gupta said.

Gupta and Procaccia both told us that if you know what you want to do, coding is not the hardest part.

(Image: DianaHirsch/iStockphoto)

(Image: DianaHirsch/iStockphoto)

Implicit in crafting the algorithm is "worst case" thinking -- how long will an algorithm take to run if given a sizable input? Procaccia illustrated the problem with the following example: What is the shortest route between New York and San Francisco? Drawing a straight line between the two points is easiest. Using a road map can be a little more difficult, but the Interstate Highway System provides a ready-made route that is pretty direct. Writing an algorithm that uses state and county roads only would be more complicated.

"Labeling the input here, that part is the easy part. The hard part is understanding what to do," Procaccia said. "The more elegant the idea, the more intuitive it can be."

Once you have a satisfactory algorithm, said Google's Marshall, it gets filed in a library for software engineers to reference. Should another project arise that is trying to solve a similar problem, that filed algorithm can become the basis for another solution, Marshall said.

It's also a good idea to revisit algorithms and "re-factor" them for improvement, Marshall said. While performing that re-factor is an ideal best practice, in reality, it's not done often. "If it is not broke, don't fix it" is an operative phrase for many developers.

Search queries propel a lot of Airbnb's business, as they do for Google. At Airbnb, refining the algorithm means looking at search results and looking for the conditions that produced lousy results, explained Gupta.

Gartner's Sondergaard in October described the algorithm economy as being about the "density of contact" with "your business at the center of a network." Whether you are talking a web of customers or a Web of millions of devices or "things," your business is a network of algorithms interacting together. Are you ready for the algorithm economy?

**Elite 100 2016: DEADLINE EXTENDED TO JAN. 15, 2016** There's still time to be a part of the prestigious InformationWeek Elite 100! Submit your company's application by Jan. 15, 2016. You'll find instructions and a submission form here: InformationWeek's Elite 100 2016.

William Terdoslavich is an experienced writer with a working understanding of business, information technology, airlines, politics, government, and history, having worked at Mobile Computing & Communications, Computer Reseller News, Tour and Travel News, and Computer Systems ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Ninja
1/3/2016 | 3:46:18 PM
Re: Algorithm and data
And let's not forget good documentation. If conditions change, especially if the original algorithm writer is gone, good documentation can make the differerence between a smooth transition or the need for starting over from scratch.
User Rank: Author
1/3/2016 | 11:20:11 AM
Re: Algorithm and data
< At Airbnb, refining the algorithm means looking at search results and looking for the conditions that produced lousy results, explained Gupta.>

That's a very important step in the process. Based on what I see in searches on many sites, they don't bother with that. It can be very frustrating to put in clear, simple terms and get back irrelevant results. 
Li Tan
Li Tan,
User Rank: Ninja
1/2/2016 | 1:44:22 AM
Algorithm and data
Alogorithm and data are both very important for a good software. More precisely, data model and algorithm should be designed/chosen together so that they fit well with each other. You cannot simply say algorithm A is always better than algorithm B - it depends on what data set you want them to operate upon.
10 Trends Accelerating Edge Computing
Cynthia Harvey, Freelance Journalist, InformationWeek,  10/8/2020
Is Cloud Migration a Path to Carbon Footprint Reduction?
Joao-Pierre S. Ruth, Senior Writer,  10/5/2020
IT Spending, Priorities, Projects: What's Ahead in 2021
Jessica Davis, Senior Editor, Enterprise Apps,  10/2/2020
White Papers
Register for InformationWeek Newsletters
2020 State of DevOps Report
2020 State of DevOps Report
Download this report today to learn more about the key tools and technologies being utilized, and how organizations deal with the cultural and process changes that DevOps brings. The report also examines the barriers organizations face, as well as the rewards from DevOps including faster application delivery, higher quality products, and quicker recovery from errors in production.
Current Issue
[Special Report] Edge Computing: An IT Platform for the New Enterprise
Edge computing is poised to make a major splash within the next generation of corporate IT architectures. Here's what you need to know!
Flash Poll