How We Built Our Own BI: When Off-The-Shelf Apps Won't Do - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Big Data Analytics
11:06 AM
Sagi Bakshi
Sagi Bakshi

How We Built Our Own BI: When Off-The-Shelf Apps Won't Do

Sagi Bakshi, GM of Digital Solutions for ironSource, needed a business intelligence application to meet all of his organization's unique needs. When off-the-shelf options proved inadequate, his team decided to build its own layer on top of an existing system. Here's how it did it -- and the role IT played in the process.

InformationWeek 2016 Salary Survey: 10 Best-Paying IT Leadership Jobs
InformationWeek 2016 Salary Survey: 10 Best-Paying IT Leadership Jobs
(Click image for larger view and slideshow.)

In our industry, a sturdy business intelligence solution is key. It helps us power our data-driven platforms, analyze valuable consumer insights, and track our performance -- all in near real-time. This last bit is especially important, considering that at ironSource -- an online software distribution and monetization company with a complete ecosystem for downloadable applications -- we process billions of data points monthly.

Our customer success managers -- who, in a way, were our first customers -- wanted to see how campaigns were performing in real-time, so they could easily adjust and optimize them on the fly.

Naturally, we looked into several major BI providers, hoping one of them would be able to address our strategic needs. BIScience, GoodData, and Qlik were among those we evaluated.

None seemed to quite get the job done. There were a few critical metrics and KPIs missing from each.

[Don't do it like this. Read 8 Ways to Fail at DevOps.]

So, we turned instead to building an additional layer on top of one of the existing BI platforms, and then adjusting it so it could support the essential business units and features we needed.

What We Did

It took a while (six months to be exact) until we finally developed the perfect BI solution for us. Because we built it ourselves, we were able to add useful and industry-specific features. More on that later.

First, let's talk about how, during the course of this project, our small BI team acted as salespeople, customer support, Q/A, and -- of course -- developers. At ironSource, it's typical for our teams to wear many hats, so we were prepared.

In the beginning, our BI team presented our customer success managers with a trial version and asked them to use it and offer feedback. It was a struggle at first to get the customer success managers to adopt the new tool.

It was an amusing role reversal -- usually they're the ones nudging external leads to use ironSource solutions. This time it was up to us, a group of developers, to get our internal clients to integrate our portal into their everyday routines.

We sent constant email reminders with functionality updates, animations, funny gifs -- anything to catch their attention. We even enlisted the help of our in-house studio to design appealing graphics.

(Image: ismagilov/iStockphoto)

(Image: ismagilov/iStockphoto)

Truthfully, what really pushed the customer success managers to adopt the platform was showing them that our upper management team was using it, too. Our leadership demonstrated how important and necessary it was for them to start utilizing our BI portal.

Specifically, I'd send screen shots pulling numbers and statistics from the BI platform to the customer success managers and ask them to check up on certain campaigns. This way, they'd need to go into the portal in order to respond to and access the data.

This helped us create a shared language for discussing performance, results, and expectations with clients, which can be key to both successful client management and company-wide unification. Since ironSource is made up of many divisions, this is a critical challenge for us.

Customer Support

Failure was a given. But as the saying goes, if at first you don't succeed, try, try again. In the beginning, as with most new platforms, there were discrepancies between the data our portal was presenting, and the actual data. Really, the only way we could fix this particular mistake was with the help of the customer success managers, who consistently flooded our email inbox with bugs and requests.

In fact, within a few days, we set up a WhatsApp group for support, so users could direct all their concerns to it, instead of emailing people individually. Customer support was critical. If it was 3:00 a.m., my team was up and ready to fix bugs ASAP. After all, ad campaigns don't sleep. If something was off with our numbers, it couldn't wait until morning. It wasn't uncommon for our team to stay in the office until midnight during those days.

Quality Assurance

Unsurprisingly, our initial version needed quite a bit of work. In the WhatsApp group, the customer success managers outlined three major flaws:

  • The user interface was confusing
  • The platform needed a faster response time
  • The platform also lacked some key metrics

First we tackled the UI, dividing the various different dashboards into logical groups and adding intuitive icons. Our design studio was a great help here, working with us on logos, fonts, and general design. They're UI experts, so, luckily for us, they took on most of the burden on this front.

Next, we tweaked the response time. Our platform holds millions of data points, so it needs to work fast, otherwise it becomes unhelpful, and the customer success managers fall back on using spreadsheets.

Last, we enlisted the help of ironSource's analytics team. Its members helped us calculate the complicated metrics our customer success managers requested. For them we developed hourly and monthly campaign tracking.

Sometimes customer success managers need to create monthly reports to demonstrate progress over time. To accomplish the task, we used Pentaho for ETL, which allowed us to aggregate millions of "by the minute" records into "by the month" ones.

In addition, because of our volume, we knew we needed to compare hour-to-hour results. If there's a 30% drop within an hour, the solution alerts the team so we can react quickly.

Calibrating the alerting is a fine art. On one hand, we don't want to send too many alerts (and cause alert blindness). On the other hand, we don't want to send too few (and risk missing important events). In fact, the KPIs and features we've added to our BI solution for internal use are now being used many of our external customers as well.

Of course, this is an ongoing process. We develop a new version once every two weeks based on feature requests by our own customer success managers and our external customers. Or, if a dashboard isn't being used, we remove it, try to understand why it was underutilized, and learn from it.

Where Does IT Fit in?

Our IT team was in charge of preparing the information security architecture for our BI portal. At an early stage, we had to decide if we wanted the BI portal to be available outside the office. It made the most sense to do so. After all, our customer success managers need to have access from home, because data is in real-time.

In addition, once we extended it to our external customers, they needed to be able to access the portal from all parts of the world, and in all time zones. Of course, with this decision came many security concerns.

The IT team developed a secure system to allow external access to the portal. Once it was built, they ran stress tests, vulnerability tests, and penetration tests to make sure absolutely nothing could attack and harm the system.

In addition to security management, the IT team was vital in designing and assessing the hardware requirements of the machine that runs the portal's servers.


In time, we became a well-oiled machine, consistently receiving feedback from the customer success managers, assistance for implementation from the analytics team, and security maintenance reports from the IT team.

Usage of the BI platform jumped from 200 visits a day to 1,500 in only six months. (There are about 250 users on the BI platform from the business, finance, and R&D departments at ironSource.) Today, the average customer success manager accesses the platform 15 times a day, even on weekends.

In the end, building our own BI solution was perhaps the best decision we could have made. Not only did it benefit us and improve the way we tracked our own performance, it also gave rise to a new solution for our external customers. This way, they're able to use the same high-level measurements and metrics we're using, all while being able to easily interpret and understand complicated numbers on an easy-to-use dashboard.

What's your BI story? Have you attempted to build your own? Are off-the-shelf solutions meeting your needs? What role is IT playing in your BI efforts? Does adoption of a new BI solution always require a push by leadership? Tell me all about your experiences in the comments section below.

With over a decade of experience in the industry, Sagi Bakshi is the GM of ironSource Digital Solutions. He has been with the company from its earliest days. Sagi built and nurtured ironSource's platform for software developers, transforming it into ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
10/6/2016 | 4:08:33 AM
It could be really effective in a few year I hope.
User Rank: Ninja
10/5/2016 | 1:44:33 PM
IT Defintion
I got a kick out of part that "IT" contributed: infrastructure and security. I would argue your entire company is IT, with exception of certain back office jobs like A/P, accounting, etc. 

Whether you realize it or not, much of that entire project falls under IT, from any traditional definition in a manufacturing type environment. When you are a tech company, I don't think it redefines what IT is. Just means the "business" employees are extremely tech savvy, maybe even having Comp Sci backgrounds. 

For example, I'm a developer at small business unit. I have to manage the entire spectrum of a project like that. I probably would have asked 3rd party experts to assist with security and infrastructure stuff at the scale you are operating. But that doesn't make them "IT" and me not. 
2021 Outlook: Tackling Cloud Transformation Choices
Joao-Pierre S. Ruth, Senior Writer,  1/4/2021
Enterprise IT Leaders Face Two Paths to AI
Jessica Davis, Senior Editor, Enterprise Apps,  12/23/2020
10 IT Trends to Watch for in 2021
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/22/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll