Big Data and more statisticulation


by Irving Geis in How to Lie with StatisticsToday’s New York Times made my hair stand on end. Kenneth Chang’s piece,  Parsing of Data Led to Mixed Messages on Organic Food’s Value,  admirably points out how easy it is to confuse the public. The article raises good questions about research methods, standing by institutional reputation not merit.  I wonder  Is this  just another sign of academic practice needing a bit of a makeover?

The Case in point

Stanford, whose sterling reputation lifted higher with news celebrating its latest Nobel prize winning economist, added new fodder to the debate of organic vs. traditional agriculture methods and practices.  The respect enjoyed by Stanford university and its published research shuts down public questioning and personally, reminded me of Darrell Huff‘s 1954 slim volume How to Lie with StatisticsA sentiment also captured whimsically by  Irving Geis in his  illustration(shown on the left)  in the same volume.

Huff coined the phrase “statisculation,” to refer to those who use statistical material to misinform people.  Chang’s article doesn’t judge. He merely helps  innocent readers understand something about meta-analysis, a perfectly acceptable method that produces very useful and cost-effective insights by re-examining other researchers’ data. Also customary in scientific circles, the Stanford published results and conclusions, raise questions and challenges pressed by other researchers.

The problem is that this dispute and airing works for closed loop academic circles where everyone understands and accepts the evaluation process, and accepts that knowledge and acceptance never comes with only one paper.  Chang’s article points out that Kirsten Brandt, the scientist who led the England Newcastle University published 2011 study  though also a meta analysis, produced very different results, did not enjoy the wider public audience and hit the social media airwaves.  Consumers  making daily buying decisions instead are made vulnerable by sound bytes shared by those who stand to benefit the most from the Stanford University researchers’ findings–that organic doesn’t offer material differences.

On average Stanford’s findings may be true, but since when do we actually care about the average?  The tomato I buy isn’t average, and how, where and when its produced changes its quality.  For example, this year the drought didn’t produce the same volume or quality of tomatoes locally as last year. Chances are the nutritional quality differs too. So why would averages across geography and year prove meaningful to compare?

Big Data and its ongoing hype does offer  greater opportunity. The capability to look at and analyze more attributes, makes it possible to discover more relationships. produce new insights, increase our understanding of subtle differences into the relationship between things or ideas.  The cost/benefits of this explosion of hype around the promise of big data and analytics however in the short-term may prove disproportionately beneficial.  As the Chang article illustrates, more data doesn’t improve the analysis quality. Limited capability to test and engage in the healthy scientific defense process, may lead to greater manipulations or statisticulation.

Few people capably and competently understand differences in error terms or appreciate that some reported differences just aren’t as significant as others because the method relied on averages and not the full range of the distribution. If I’ve already lost you, then perhaps you should take a look at Darrell Huff’s book.

Required reading for any published author and every student is a basic primer in statistics.

 

Advertisements

Discovery Needs more legs


 

 

As a kid, I remember how much I loved Cracker Jack.  Sure, the caramel flavored popcorn and peanuts were tasty. But the discovery that every box had a prize inside hooked me.

Discovery, the process of realizing something new doesn’t have to happen joyfully. Discovery associates or naturally links in our mind with surprise, so why not celebrate the unexpected emotion, object, or idea too?  Considered as a process, discovery advances our understanding of our environment– its conditions, dimensions, contents and workings.

Discovery is how we learn

Yesterday, the United States celebrated Columbus Day.  I asked my 8-year-old niece what she knew  about Columbus.  She told me he found a new way around the world and “found” America.  Over lunch she and her older 14-year-old brother shared what has become the takeaway on the Columbus story.

Since I was in grade school, the basic story has undergone serious revisions.  Columbus shows how facts change when we acknowledge the legitimacy of other perspectives..  It’s reasonable that the language of the story would change based on your point of view and that additional information might challenge previous assumptions.

In suburban Chicago kids learn about Columbus for the first time in 3rd grade and by middle school they learn Columbus was lost, as in clueless. The facts we know today make it impossible for Columbus to have “discovered” America, particularly when he mistook for Indians the native inhabitants, which further exemplified that Columbus was dumb.

The turnaround of the Columbus story, from a brave, invincible, enterprising captain and discoverer of a new world claimed for Spain, saddens me.  My nephew further explained that  Columbus wasn’t brave, he was headed for jail unless he stepped up to take the voyage .  Really?  This was a new twist on the story, until I realized that my nephew merely turned around the order of the events in the Columbus story.  Columbus did end up in prison toward the end of his life, and may have died there.

Still, were it not for his persistent belief in his calculations that a faster route lay to the west, he wouldn’t have pleaded in royal court after royal court from Italy to Spain for the chance.  It still took courage, leadership once Queen Isabella  granted him three boats and paid for his crew to sail west across the Atlantic, a direction few were willing, and fewer succeeded.

Discovery, tactics to change beliefs

The fuller story of Columbus, the sum of all his activities in life may not suggest heroism, or show signs of a dignified, inspirational, gentleman, warrior or leader.  Kids learn the facts and left to balance them and maybe that’s appropriate.  Discovery is like that, sometimes our first beliefs of people or things shatter with additional perspectives, information, facts and context.

Nike the company, initially revered for its shoes was later vilified when buyers learned the conditions of the factory workers overseas.  Its founders made changes and once again Nike stands tall and stronger than ever. Apple has not yet had the same comeuppance, in part because few people are ready, willing or interested enough to look past the iconic reflections of the brand and its products. Discovery is like that too.

We like what we know and how  it makes us feel.The less I know about Apple, the easier it is for me to feel good.  I can be hipper, cooler with my Apple manufactured technology. Why do I want to know how Apple makes its products? Why does it matter?  Besides, will it suddenly reflect badly on me if I know what Apple does that’s bad and ignore it?

Imagine the SuperPacs using their political campaign tactics to pelt Apple for disregarding their overseas contractors’ working conditions and human rights violations.  Condemning those who profit from another’s misfortune might come back to haunt them.  Circulating this news might be considered shortsighted and run contrary  to the personal Super Pac investor and their interests.  Discovery can be selective.

Discovery and firsts

Children in discoveryDiscovery, doesn’t mean necessarily first. Discovery when nurtured, encouraged and celebrated rewards learning.   Successful businesses uncover markets for their products or services, and yes, I’d prefer business run responsibly but opportunity doesn’t always wait for all perspectives to align.  A responsible discovery process and attitude serves to remind every one of  possible consequences, as in “don’t cut off the hand that feeds you.”

In any first encounter, natural instincts raise our defenses.  In The prisoner’s dilemma model, the consequences of cooperating and not cooperating are very clear.  The dilemma presented reflects an inability to take into consideration what choices others have made before deciding whether to cooperate.  The model demonstrates the complex relationship between the learning and discovery process and our choice of actions. Another model is the Monte Hall problem in which the discovery of more information challenges our ability to win.  Both of these models illustrate that it’s not discovery that presents the problem, but the unsettling feeling of uncertainty that sometimes follows.

The absence of information, may represent a self-imposed limitation. though some problems are more complex and lay outside the bounds of present knowledge. Gaining more knowledge merely invites us to integrate it with what already know, and not dismiss it outright.  Open discovery processes increase possibilities for everyone, and we should begin to help others appreciate and respect discovery more if we want to play with more favorable odds.

Open minds filled with a sense of possibility, make them more accountable for their actions.  Columbus was ultimately accountable to the monarchs that invested in his voyage of discovery.  So the captives he took landed him in prison, impoverished in spite of the riches that resided in the lands he discovered.   None the less, we should do a better job of helping our children understand the value of Discovery and use Columbus day to celebrate and  promote adaptive, innovative thinking.  I hope that’s why Chicago Ideas Week began on Columbus Day, if not I suggest it become a tradition!

Can you think of other examples of the value of open discovery?  Please share them.

 

Are you ready for the honesty in the mirror?


It appears that Social Media shows signs of maturity and its appetite can be fierce if not fearsome.  A year ago, the Arab Spring signaled the power of social media to rapidly help like minded individuals with shared frustrations meetup and act in solidarity against institutions. The speed of the social sharing had been seen before but not on this scale and not for this serious a level.

I happened to come across two examples in which social media has begun to provide real levers of change,and illustrate what Scott E. Page, at the University of Michigan and other behavior modelers, recognize as the power, not just the wisdom of the crowd to change outcomes. 

When I say levers, I’m talking crow bars, not mere  shoe horns.

Estimates by the Pew Reseach Center Internet American Life project show that  “two online activities that are nearly universal among adult internet users, as 92% of online adults use search engines to find information on the Web, and a similar number (92%) use email.” Shoehorn persuasion tactics. Email being the easiest and direct persuasion tactic. Its free and so the more they write the more they hope I’ll respond.  Crowbar tactics are when the audience flips the lever back on the sender.  This isn’t the same as the Arab Spring, or if I’m wrong I hope you’ll tell me.

Case #1 Candidate Accountability

Without the benefits of stats, I’d estimate the savvy of the presidential campaigns’ use of social media –getting out their messages to voters, and pulling in donations and volunteers–even with the public and niche user communities’ scrutiny of the candidates claims.

Its human nature  and certainly consistent with the American capitalist tradition to expect something in return, the exchange of effort and value.

Candidates have always been aware of the double-edged exchange of access to voters. More importantly, they also need influence to get favorable results.   I live in Chicago whose rich history of machine style politics used patronage as reward to its soldiers who helped register and then turn out the votes. Social media in 2008 was young,the candidates who took it by the reigns and broke in the users learned how the new media drove influence and when relationships mattered.  Pelting your audience with emails needed a little finesse, to prompt sharing.  Give them something worth sharing and your audience will provide the influence. The voters would be the muscle standing behind the message.  Remember Sarah Silverman’s hit video, that encouraged grandchildren to go visit their grandparents in Florida and get out the vote for Obama?

Again, I’ve no stats, but Jon Stewart’s team as well as Stephen Colbert among others at Comedy Central, provided ample video clips worth sharing.  They also did something else. They reminded their viewers that, like Dorothy they too had the power all along to take on a candidate’s claim and hold them accountable.

Fast forward to 2012.  I receive a ridiculous amount of email pleas for donations and offers to win a chance for dinner with the president.  It’s a little like publisher’s clearing house, and I don’t care about their fundraising deadlines.

Here’s the case that John Souza writing for Fast Company yesterday mentioned.  VP candidate Paul Ryan appealing to potential voters shared, maybe even boasted that he had run a marathon in 3 hours. The effort intended to build solidarity, open up  and reveal a little about him.  But remember this is 2012, and social media’s two-way lever, makes the public more powerful.  Sure, a candidate needs to appeal to the voting population, and leverage common affinities to his favor. How did he fail to realize that this audience had pride in their efforts and might take offense? The savvy campaign staff didn’t anticipate the indignation his boast would draw, nor that it would prompt greater scrutiny.  Rather than drawing them close he found himself suddenly at a huge distance.  [Read the fuller story here, Obama and Romney’s Biggest Social Media Fails courtesy of Fast Company.]

Souza identifies three signs of social media’s crowbar power all made possible by people’s ability to find and then share content that they ultimately can’t control.  No surprise when Facebook has 1 billion users, or  92%, and growing, of american adult internet users use search and email.  Souza points  out how  social media “by shining a continual light, from every possible direction, on every move a high-profile candidate makes,”  has made it impossible for candidates to control their campaign and message.

Elected officials don’t get any more of a break than candidates.  Arguably, few institutions are beyond the reach of social media.  Anyone rewrite the updated version of the Emperor with no clothes yet?   Which brings us to case #2.

Case #2, the messiness of governing.

In early September, the Chicago Tribune launched a useful interaction with the crime data that the City of Chicago has made public. I dropped by the local Datapotluck meetup and met the staff responsible for its launch and learned more about the data challenges.   Six weeks of 3 programmers full-time and the results are very impressive.  I especially appreciate the context they went out of their way to include.  Judge for yourself Crime in your community.

Great that they are more eyeballs analyzing the data, that’s one value to the city.  But more eyeballs analyzing the data is also what a) prevents the mayor’s office from providing all of its available data in real-time; and b) sharing the parallel data of police responses.

What happens when we make people more aware?  Joe Germuska who designs the news applications for the Tribune,  shared some of the insights gained in designing the crime site by sitting with people in various communities to understand what data was useful and why. Parents who want to insure the kids got to school safely use the interactive maps to plan the safest routes. Or the context alters the image of a community area when you the crime stats appear in proportion to the population.

Getting more people to think and act is one of the oldest games on the planet.

There’s no place to hide anymore, and whether you share the data or not, people will hold others accountable.  That  return lever has always been there, but rarely exercised by lesser known individuals raising their voice, surprised to find it carried by strangers. We all benefit from learn how to use these tools more effectively, to motivate, incent or  engage people constructively not merely destructively.

Democracies are messy, but the shot of more people benefitting increases the more transparent and visible the decision-making process.  So Rahm, good job on learning and sharing but you’ve got more to do!