Optimizing on the Question – Big Data Analytics

In life there are opportunity costs. I have spent many a Sunday fretting over whether to clean the basement, watch the game, or spend the afternoon on single track with one of my kids. We unfortunately can’t create a duality that allows us to experience two simultaneous events, I always have to choose across my options. Companies face opportunity costs too. Do you hire head count for Brazil, reduce inventory in mid west, manage COGS, or acquire a company. There are constant decisions that leave alternative opportunities like dust on the floor. Matter of fact, companies are really layers of opportunity cost decisions we call strategies.  Strategies allow us to plan and execute with focus. Sometimes strategies are new and/or overt; sometimes they are organic and implied (a kin to culture). Usually companies have an agenda that is known by most and for this conversation I will call it a strategy. I would say for the majority of companies, strategy isn’t always democratic and out of the box for everyone. There’s a bit of trickle down defined and disseminated. “This is a growth year for us, we need to penetrate the market…”, “We’re concerned about Europe…”, “Divisional goal is 500 million…” or something similar.

Many of us, at various levels in the organization, are bounded by constraints and are asked not to create and advance the cause aimlessly, but to a set of criteria. This is important to align the organization and execute with powerful sameness.  Yet once in a while something happens, the market changes, new tech shows up. When this happens, the technologists get geeky and the C-level gets hungry, scared, or both.  Something “new” arises from the corporation that breaks the rules, and clears paths for new ways of thinking.  Think about the Dotcom wave, it is currently the quintessential example of this scenario.  All of a sudden, 20 year olds were becoming executives, Dotcom companies had valuations that surpassed blue chips, and people who sold socks to dogs were getting millions in VC money. The “rule followers” were punished and mocked as dinosaurs. The companies considered long-standing cultural strategies as broken and many chose to replace process while revolutionizing their offerings.  This worked for some, destroyed others, and embarrassed many in the ranks. We’re now entering a similar wave around Big Data Analytics.

My term for this wave/bubble/era… is Big Data Analytics (BDA). I use this inclusive term for the wave of change that is taking BI and departmental analytics and merging it with social media, global mobile user, cloud, and various forms of big data. Here within this wave again we have many great examples about how knowing more about us (from what data exists around us) allows for new insights and is ushering a new age. There are examples like “The Human Face of Big Data” (#HFOBD) with Rick Smolan, EMC/Greenplum Analytics Workbench on a 1000 node cluster developed to study big data (i.e. twitter, facebook, etc). or SAP’s recent “Real-Time Race” (#RealTimeRace) pitting two System Integrators against each other on stage developing live solutions on HANA. These bits of news get us all thinking “what if”.  However, as a large group of capitalists, I’m not sure we know how to leverage it, how to make decisions around opportunity costs for BDA. We’re in a bit of an innovation bubble and the ultimate question is what can we do to improve and prioritize our choices?

I have a few recommendations for everyone to consider in their business. It’s a short list of 3 things that I believe the smart companies will consider either organically or by reading this blog 😉

1)      Data is the new Information Technology (IT). Read “The Big Switch” by Nicholas Carr and you’ll see that since the birth of the “industrial company”, there has been a “technology” group which consists of smart, well-paid resources who apply the latest tech to business. First it was electricity & machines, then business systems, then computers, then data centers, and I propose the next wave will be whatever Big Data Analytics becomes.  (I would call it “Knowledge Management”, but we already blew that logo in the 90’s…) What this means is BDA will be pervasive and inject itself in many aspects of business, not just creating opportunity to increase revenue through traditional means.  Companies need to build new disciplines that identify and develop data use. Picture the 1960-1970’s. We used vacuum based mainframes, typewriters, rotary phones, and business men took 2 hour martini lunches. The world of iphones, tablets and angry birds is very different. I believe our future world will be thoroughly basted with data driven wisdom which will have even larger impacts.

2)      A Corporate Decision Strategy is needed. How we make decisions and what questions do we ask? We need to be much better at asking questions than we are today. I am under impressed by the long line of shopping basket, alternative offer, or Twitter sentiment studies I see.  This is applying old perspectives of your customer to a field with much greater opportunity. How do we involve crowd sourcing, self-service, social media, and data analytics to change the customer experience, the corporate workforce, and ultimately what is considered corporate core competencies.  I think a great way to start is a review and inventory of BDA capabilities within the business and an assessment of how questions are asked that create data analytics projects. Questions and decisions don’t serve the strategy, they are the strategy going forward.  Also note, the people who can define the right questions are more like artists, musicians, song-writers.  Many know there is a strong correlation between scientists and musicians. Similar to how computing jobs became loaded with creative people, we’ll need to migrate these skills away from programming and towards question development and decision strategy. Good news, the new recruits have grown up as “gammers”, this plays well into foundational skill sets needed.

3)      Change Business Systems – I’ve written assembler code and I’ve run analytics models. They are both very difficult to conceptualize and navigate. It’s a relatively small group of individuals that can execute these skills and thus create a real problem of scale. AI and machine learning are the beginnings of techniques that begin to help the average person to become a contributing member in this new age. We need decision support within BDA to morph from manually running a “K Means” model or hand developing “ordered pairs” to something less academically rigorous, and data capture/management/cleansing to become more intuitive and automated.  If we can automate “Mario Kart” as far from assembler as it is today, we can do the same in the era of BDA.

As companies refine and develop their BI/Data Analytics programs into the era of BDA. I think there’s an equivalent need to rethink the questions they ask. Do we need less sales execs? Do we need more data security?  Do we want to know what else we could sell someone buying running shoes, or do we want to help them design a new shoe specifically match their demographic/health needs? Can our customer sell themselves? And if so, how do we need to change what we sell to be competitive? What are we good at? What companies should merge to exploit inherent opportunity?  These are the opportunity costs of the new era that will emerge from BDA. To gain insight from data, we must first ask the right questions of it…