Manage your opportunity: Mine and Mind wider meanings

It’s not just a generational thing, there are so many more places to go than any of us have time or interest. Oh, I just meant on-line. Whether you do or don’t bother with Facebook,  or seek wisdom browsing  social sites, every place on-line data mining traps lie in wait.  Search tools once brought great excitement. At the tip of your fingers, one box combined the knowledge of encyclopedia, yellow pages and the shopkeepers of the world. Today, I wonder,  search and its promise of the information revolution resembles a trap and less a portal to discovery.  Personalization and customizing the experience takes priority over serendipity and pure exploration. That’s the clarion bell signaling opportunity and let me share my own cursory take.

English: The three biggest web search engines

English: The three biggest web search engines (Photo credit: Wikipedia)

The trend lines all show mobile apps displacing Google search. By the way, so do my browser’s add-on buttons. Still, old habits are hard to break. Bing and Yahoo, sure they keep trying, but Google’s continuous upgrades still deliver  personally satisfying search results faster. Their under the hood tracking of my past choices and search history increase their predictive precision.  I’m a statistical geek at heart and so I miss the old Google, that shared the probability score on every result. Today, for kicks I typed “Yahoo” into my browser’s Google search box. and these numbers appeared above the search:  1,200 personal results. 4,430,000,000 other results.

Ironic, to discover 4.4 trillion requests made to Google to find Yahoo doesn’t it?

These numbers illustrate Google’s additional value–their search provide me an opportunity to put my search into a broader context.  In this case the numbers led to an insight, how I can quantify and size a search word’s popularity. I can measure a meme or popular thought’s magnitude. I was on a roll, what other meanings or associations might these search placed terms hold?

My browser (Firefox) search box dynamically lists offers suggestions to help me refine my search.  As I begin to type, additional search terms appear ranked by popular preferences. The phrases anticipate  and nudge me to narrow down my intention, refine the Yahoo search from 4.4Trillion results.  The drop down menu offers 10 suggestions: mail, (the address that I’ve used to get my mail), finance, news, sports,  answers etc.

Hmm.. imagine using the old yellow pages. Organized by category, the listings appear alphabetically.  Google and the rest of the search engines never worked that way. If I try, using the category Restaurant, I get

1,250 personal results. 364,000,000 other results

What do these numbers mean?  Looking over the first few results in the search output  and accompanying map, I’m shown a neighborhood where I used to live 30 years ago and visit occasionally. Before wondering why that neighborhood popped up I compared Google’s competitors.

Bing returned 50,100,000 results and Yahoo reported 48,500,000 results. These are not small discrepancies and may explain why Google remains #1 not just in my preference stack but apparently for the world too.  But there’s more.

Ambiguity the new opportunity

The habits Google encourages and its customized learning of personal preferences  revolutionized how people spend their holidays, shop, work and play.  For the last several years Venture Beat reported in January that Google grew its “semantic network” to at least 570 million objects and 18 billion facts .

In the hands of marketers, the more numerous, diverse associations attached to an idea or phrase makes good business.  The variation allows room to play off nuanced differences and at the same time drill down to find the universal or shared meanings that bring different community segments together.  Once established, shared meaning offers a foundation upon which new experiences and associations can be built,  It can also segregate individuals into subgroups based on mutual understandings. Consider the difference between  word slang  and its normal usage as in  Bad. What associations come to mind, reflects all the nuances of your interests and the company you keep. The same word carries multiple definitions and its usage varies within different populations.  Google ad words and SEO allows marketers placement based on the nuanced choice of their target market.

So how did Bing challenge the space  or more importantly what inherently does its value proposition offer? Bing displays results in three distinct columns: the traditional search , a second column of paid advertisers and the third Facebook’s search results. It may be nice to differential but I’m not sure I understand the benefits.  Or if I did, I’d use Bing more.

Historically, the search service results listed in order of frequency of association or popularity. The resulting match returns ranked sites whose keywords had greater influence in  that particular domain. To effectively compete, I’d need to use the same key words to get my ideas on the inside of information sharing circle.  This is the old Buzz game, I want people to talk up a particular topic or a brand, the idea needs to insert itself into the conversation, right? Google still helps you see the trend lines behind the scenes, which words are trending over time, when and where.

Extrapolation from the past ain’t  foresight

Which approach develops foresight? How can someone track the spread of ideas or get a true bead on what’s coming next?

Yes foresight not insight, as in opportunity creation and positioning for advantage. Tools to find and understand emerging customer trends requires something else. It requires context. Let me introduce Crawdad Software. Their process follows a  patented system called Centering Resonance Analysis (CRA; Corman, Kuhn, McPhee, & Dooley, 2002). It  differs from traditional inquiry methods that deliver results based on word frequency. CRA’s Latent semantic analysis,”uses computational linguistics to model a text as a network of words…its grammatical rules understand how words take meaning from context. Whereas word frequency methods create insight based on a “pile of words”, CRA creates insight through applying network analysis.”  The following is an example of Aesop’s fables courtesy of Kevin Dooley, CEO of Crawdad.

Network analysis?  

Which matters, who does the sharing or what exactly they share and why. Which words they use turn out to be useful to understanding and predicting reactions.  That’s the association that triggers action.  The map above illustrates how naturally Aesop comes to mind when the word fable appears.  But notice how many additional words appear too.

Imagine a leading manufacturer’s #1 product, also leads in its category, suddenly loses ground to another competitor. It failed to notice an opportunity and  others stepped in to their space. Foresight capability, like radar, offers early warning signs of a new attraction or distraction drawing interest in your field of operation.

Once people talk about an idea, understanding who does the talking is as important as why there’s talk at all. Semantic analysis techniques assess the context, meaning and relational significance of this new idea. Whether outside or inside, social media listening posts require additional intelligence to be useful.  Sure, there’s added value to Google’s intelligent semantics.

Example of search term semantics

Bear 622,000,000
Bears 266,000,000
bear with me 276,000,000
bear witness 7,560,000

Its methods borrow from Crawdad, but its output sure doesn’t.  I assembled the table above manually.  In March, I discovered  IBM Data Analytics offers semantic sentiment analysis along the lines illustrated above. They  plan to put the results in the cloud  this summer and make it more widely available, or at least platform neutral for subscribers. So I am not able to create a comparison for you,or even an illustration.

All of these tools offer greater opportunities. Numerous data-mining tools, and increasing integration of social media into knowledge management functions, dashboards and automated evaluation systems makes trivial the opening description of what’s going on with search.  Getting a bead on what’s next  however won’t matter unless you are capable, resilient and flexible enough to adapt and make use of that knowledge ahead and better than others.

The understanding of diverse associations makes sense to marketers who understand what to do with them. The challenge is to help more people within your organization understand what to do with this information.  IBM shared these competencies at their Analytics summit last week.

Social Media Analytics is about Business
Marketing   Human Resources   Risk Management
1. Brand Reputation 1.Company Reputation 1.Partner Reputation
2.Messaging 2.Attracting key professional talent 2.Union members wants
3.Campaign Management 3.Attracting College talent 3.Identify key managers online conversations
4.Competitive Positioning 4.Identifying key reasons for attrition 4.Reputational risk
5.Identification of key Influencers 5.Identification of key Influencers 5.Impact of my customer’s reputation


The world doesn’t really get any more complicated, just more diverse. We keep adding new words, new phrases and at the same time, adding new meanings to old words and phrases. Survival depends on proper interpretation, how well we understand others and how well they understand us. I suggest there’s great value that we leave on the table when we let ambiguity get the best of us.  Take time to verify others understanding, don’t merely respond based on assumption that is unless you plan to seek forgiveness every time.  Start listening fully first, learn what others understand, not just what but why their expectations exist and then choose whether to adapt or to share back.


Big 3I competencies: Why are they so darn hard to acquire?

Creating value and organic growth opportunities requires uncovering opportunities often hiding in plain sight. Innovations challenge expectations including possible returns on the effort.  We take for granted what’s under our noses even though it may be exactly where we need to pay closer attention. Understanding how perception affects our preferences makes compensation possible. Vigilance helps,  especially awareness of value on multiple dimensions. There’s a monetary aspect and there are ideas we hold near and dear.  Both values motivate human behavior and that’s what makes life interesting.  Let’s begin our exploration  looking at traditional expressions of value  after an introduction to the concept of “fundamental attribution,” or first perceptions.

Prior knowledge separates surprise from distraction.  A sudden unanticipated event will jolt our senses. Our sudden vigilant state will recede when we recognize familiar people, or cues, associated with things we know make us happy. Surprise includes circumstances or context that make us expect what comes next and so we relax our guard. The fundamental attribution idea literally draws on internal experience. Stored knowledge takes care of us, finding a fit to situations and environments we meet. That doesn’t mean we pick the best fit. Often familiar,  frequently used ideas come to mind faster. Logical or rational alternatives follow, too late to be useful. That’s where intention, pausing before reacting, offers the pre-frontal cortex time to process. This internal tradeoff makes humans wonderfully complex and predictably irrational.

The trick is to understand how circumstances get people to do what you want and avoid them blowing up in your face.  Psst, the answer goes beyond data analytic competencies, though that’s important.

Perception and preference the Big What?

Data comes in one flavor, but tastes differently to consumers than it does to product and service providers.  Everyday, more code and identifiers amplify specific and ambient details associated with activities such as tracking goods, service use etc. The convenience, cost and time savings provided by standard identifiers like bar codes, account numbers, social security numbers, email addresses and phone numbers also simplify providers, up and down the supply chain, catering to our unconscious preferences. Every day, we compromise a little more of our privacy and anonymity in the process.

The sheer volume, veracity and velocity of all this raw, “Big” data makes navigating the future possible. The tricks require exploring past and present relationships between variables. Predictive Models use that deeper understanding of variable relationships  and their interactions to create opportunities, control risk producing conditions and optimize sources of marginal profit. The results enrich our lives and few of us feel oppressed by this Business Intelligence (BI).  Big Brother does exist, but so does Big Sister, Big Doctor, Best Friend, Old Roommate, Big Pen Pal etc. In other words, government  surveillance creating the old FBI style dossiers, pales to the knowledge stored about you by your bank, Google, Facebook, Amazon and other retailers. Healthcare regulations and practices preserved the privacy of your information, and their slowed migration to electronic medical records. Their failure to keep up with the wider digital data practices have also slowed  diagnostic advances and cost saving opportunities.

Real innovations begin with insight, once the province of small tests and strictly the domain of human intelligence.  Today Big Insight crowds out the spotlight occupied by BI. Cheap storage and faster processing makes data mining possible for anyone, but it is the strategic opportunists  with the foresight to be serious players and accumulators that continue to change the world.  Recently, GigaOM  identified several use cases  while highlighting Terradata, the makers of the first terabyte scaled database. The full list is worth reading, as I mention only a few.

  1. Steve Jobs infamous statement that Apple doesn’t do customer research no longer holds true.  Terradata named Apple as its first customer to exceed a petabyte of storage. Apple rapidly accumulates  transactional information on their customers to understand customers across product groups.
  2. WalMart’s data processing and analytic capabilities go beyond simple sales efficiency. The data helps instruct and educate its suppliers with insights about packaging dimensions as well as shelf space location etc.

Intelligence to Influence requires insight

The ongoing arrival of new technologies and embedded tracking codes continue to fuel the race to understand and use real-time ambient data to influence transactions. More data makes it easier to see deeper underlying patterns more clearly.  With greater awareness, trends can be spotted and tracked more readily and the impact of different interventions tested simply and more thoroughly.

Understanding the data requires more than iterative recombination, it takes expertise. With knowledge and experience patterns can be understood by both people and machines (see Earlier post: understanding-aint-believing-and-yes-there-are-economic-consequences).  But it takes  curiosity to explore different dimensions and generate insights.  Here are two different takes:

Luis Arnal of InSitum explains what holds back many of us. Please listen to his Design Research Conference in 2011 complete  presentation, absent the charming slides. This summary doesn’t do justice to his talk, but  I wanted to share some of his key reflections and lessons on the steps to developing insights

Begin with data, or information records that represent your observations from field research. After collection, the data needs to be categorized, clustered.  Begin the analysis process using a simple scatter plot to understand the landscape or context of observations relative to the categories selected.  Using  intuition and prior knowledge, the dimensions you choose to contrast also leads to the direction in which you develop associations between the data points.  What, if any, possible connections exist?  Using imagination and creativity  lines of connection appear as  part of an effort to FIT the dots to a model.  Of course the interpretations vary. Time and patience make possible “a fidelity of meaning” and the underlying pattern comes into focus. The data’s added value  suggest patterns that slowly develop into solutions. Insights, Luis explains contain  30% Data, 30% inspiration, 30% perspiration and 10% luck.   Insights facilitate the transition from confusion to help resolve the initial problem. They are the links between what Is and What If, they help us imagine how when we don’t or can’t know.

Recent article in HBR by Thomas Davenport,  another worthwhile read, emphasizes a different set of talents and experiences.  Particularly helpful for positioning your firm is one of the closing observations about the capabilities housed within your organization and the opportunities they present.

“….their greatest opportunity to add value is not in creating reports or presentations for senior executives but in innovating with customer-facing products and processes….

LinkedIn isn’t the only company to use data scientists to generate ideas for products, features, and value-adding services. At Intuit data scientists are asked to develop insights for small-business customers and consumers and report to a new senior vice president of big data, social design, and marketing. GE is already using data science to optimize the service contracts and maintenance intervals for industrial products. Google, of course, uses data scientists to refine its core search and ad-serving algorithms. Zynga uses data scientists to optimize the game experience for both long-term engagement and revenue. Netflix created the well-known Netflix Prize, given to the data science team that developed the best way to improve the company’s movie recommendation system. The test-preparation firm Kaplan uses its data scientists to uncover effective learning strategies.”

What’s the common denominator linking Davenport and Arnal?  Both reference visual thinking or the conceptual translation of ideas into tangible representations.  Again,a  mastery difficult to acquire and beyond the bounds of computers, even those as powerful as IBM Watson. I don’t think Siri creates flow charts, but she might learn.

I did and so can and do others. When hiring for analytics teams I managed, three criteria or competencies were essential: SAS skills—statistical coding; knowledge of the business; and an ability to think through new problems. i never thought to ask someone if they could draw.  One of my teams pioneered new strategies to improve profitability.  Initially, that meant differentiating credit worthiness.  Managing the portfolio however required alternative methods to promote profitability by optimizing costs and simultaneously minimize risks.  At the time, combination of competencies we needed were rare. Above all we needed flexible thinkers to tackle complex problems  and create more sustainable solutions. We learned to bet on those who offered two of the three. In time, we came to realize that the third criteria, thinking, was one we couldn’t teach.  It became the minimum requirement. In the late 80’s, we sought out academics with  conceptual modeling experience and bypassed MBAs.  Banking wasn’t the only employers seeking these skills but we were much more flexible in hiring them.

Today, the combination of technical skills proving most valuable continue to be found among individuals who have studied complex data and demonstrate visual thinking, again not MBAs. Not all designers capabilities include assembly of a sophisticated social network analysis model, but they sure do a great job of communicating conceptual ideas tangibly.

This post began talking about value.  Should the value consumers derive match the value producers derive? Absolutely not. In business the preoccupation with return on investment makes sense for private equity focused on upside and early exit. This contrasts with Warren Buffet, who grew wealthy ” thanks to his ability to learn the value of various securities and then buy them for less, a concept at the core of value investing. “Price,” he has said, “is what you pay. Value is what you get.”

Remember the fundamental attribution concept?  Buffet’s remarks on value and his actions show how easily we mistake motive and behavior.  Companies that obsess about cost risk missing key insights.  Case in point, the recent rise and fall of JCPenney’s CEO, a man clearly familiar with the power of BIs (insight and intelligence analytics) to achieve innovation. How people interpret observed behavior matter. The more detail and the more attention to context , increases chances to uncover key actionable insights.  James Surowiecki, a notable observer of the slippery slope of over reliance on analytics, recent New Yorker column , shared comments on the widely touted and now vilified  Ron Johnson, by Mark Cohen, a former C.E.O. of Sears Canada, and now a professor at Columbia”

“In most of the retail universe, price is the most powerful motivator,” Cohen said. “This game of cat and mouse with regular, ever-changing discounts is illogical, but it’s one that lots of consumers like to play. Johnson just ignored all that.”


Playing effectively with Big Data analytics requires an unusual mix of capabilities. More than sheer brute processing power, modeling, imagining and speculating requires artistic license.  Machines will find patterns of relationship quickly, but not clear they will find the direct relationship between cause and effect. The reasons and thought processes that drive the behavior, remain domains where humans excel.

Its’ hard to believe that the same analysis that led Johnson and his team to create the square fair pricing missed recognizing coupons significance to their customers. I agree with  Surowiecki, who  suggests the impact of one  fundamental attribution created a rippling effect producing one error after another. The first error made by the board in selecting Johnson, created further error by  decision-makers and Johnson himself  in choosing  to push their half-baked strategy forward prematurely.

What do you think?