BECG theblog

My Data Science Adventure 3/ Can we please ratchet back the language?

Advanced computing is terrific, but I object to the grandiose and misleading language exemplified by the terms "neural network" and "artificial intelligence" in this context. Computing, increasingly, has given us (e.g., humans) powerful tools for processing data in quantities and at speeds heretofore unimaginable, but it is neither comparable nor a substitute for human cognition. I refer you to Yariv Adnan's superb essay, Do neural networks really work like neurons?

Just because a the structure of a computing process mimics the known structure of a cognitive process, this does not imply that it is comparable.

 

Fast Forward: My Data Science Adventure 7/

'Tis true: I have 'jumped the thread,' skipping from item 1 to item 7. I have every intention of completing thread items #2-6 with 10 months' worth of insights gained while pursuing my Professional Data Science Certification from IBM-Coursera.

In the mean time, however, I have an assignment to complete. See below - second week, item number #2, last bullet point. In the interest of completing my course of study, I am hereby producing a blog post containing the deliverables for my final capstone project. When I have cleared that hurdle, I'll come back and fill in the blanks.

Coursera Capstone Project

This is the final project for my Coursera-IBM Professional Data Science certificate program.

Here's the assignment. (Quoted directly from the Class Web site):

Now that you have been equipped with the skills and the tools to use location data to explore a geographical location, over the course of two weeks, you will have the opportunity to be as creative as you want and come up with an idea to leverage the Foursquare location data to explore or compare neighborhoods or cities of your choice or to come up with a problem that you can use the Foursquare location data to solve. If you cannot think of an idea or a problem, here are some ideas to get you started:

1. In Module 3, we explored New York City and the city of Toronto and segmented and clustered their neighborhoods. Both cities are very diverse and are the financial capitals of their respective countries. One interesting idea would be to compare the neighborhoods of the two cities and determine how similar or dissimilar they are. Is New York City more like Toronto or Paris or some other multicultural city? I will leave it to you to refine this idea.
2. In a city of your choice, if someone is looking to open a restaurant, where would you recommend that they open it? Similarly, if a contractor is trying to start their own business, where would you recommend that they setup their office?

These are just a couple of many ideas and problems that can be solved using location data in addition to other datasets. No matter what you decide to do, make sure to provide sufficient justification of why you think what you want to do or solve is important and why would a client or a group of people be interested in your project.

Review criteria

This capstone project will be graded by your peers. This capstone project is worth 70% of your total grade. The project will be completed over the course of 2 weeks. Week 1 submissions will be worth 30% whereas week 2 submissions will be worth 40% of your total grade.

For this week, you will required to submit the following:

1. A description of the problem and a discussion of the background. (15 marks)
2. A description of the data and how it will be used to solve the problem. (15 marks)

For the second week, the final deliverables of the project will be:

1. A link to your Notebook on your Github repository, showing your code. (15 marks)
2. A full report consisting of all of the following components (15 marks):

• • Introduction where you discuss the business problem and who would be interested in this project.
  • Data where you describe the data that will be used to solve the problem and the source of the data.
  • Methodology section which represents the main component of the report where you discuss and describe any exploratory data analysis that you did, any inferential statistical testing that you performed, if any, and what machine learnings were used and why.
  • Results section where you discuss the results.
  • Discussion section where you discuss any observations you noted and any recommendations you can make based on the results.
  • Conclusion section where you conclude the report.
  • Your choice of a presentation or blogpost. (10 marks)

And off we go!

Here's a link to the blog post I submitted as part of my assignment.

 

My Data Science Adventure 1/

When I finished graduate school, the MBA was all the rage. Thinking back to my earliest days working in Banking, I remember being met with skepticism: how could a background in psychology, research, and statistics possibly be relevant to business of investing, of developing products, or of selling them? 

Now, it appears, the world has changed... And I, for one am happy about it. 

For 20+ years, I quietly built a career based on the knowledge that money was invented by humans, and that economic behavior is a subset of human behavior. In so doing, I learned that:

• Research is a powerful tool for understanding behavior; that
• Ethically designed, well-executed, data yielding large amounts of data,
• Subsequently subjected to rigorous, quantitative analysis

... constitutes a powerful tool for determining which path to follow.

Measurement Matters (Part 1 of Many)

Have you ever participated in a survey or research study? It's hard to imagine that anyone would not have. Survey results cover the pages of most newspapers, Web sites, and just about any other medium. Most of them make me sputter in horror. Why? No, they're not scary. They are merely scarily bad - meaning that there are fatal flaws in the design that render the results DOA. Alternatively, the data are "analyzed" by people who don't know the difference between good and bad quality data, or do not realize that there is more to data analysis than knowing how to do the math.

Here's an example.

In December 2011, a study was published in the proceedings of the National Academy of Sciences, purporting to show that the sound made by valuable, old Stradivarius and Guarneri violins was no better than that produced by brand-new, stainless steel violins. Articles subsequently appeared first The New York Times ("In Classic vs. Modern Violins, Beauty Is in Ear of the Beholder") then in Ars Technica ("Old, million-dollar violins don't play better than the new models"). The story was picked up and featured everywhere from NPR to The DW Academie. 

There was just one small problem: the study was so badly done, and the math used so inappropriate for the tiny, non-random sample (21 people attending one conference), that there was no basis for drawing any conclusion(s) at all.

As I wrote to Ars Technica at the time:

.... the Big Mistake was to publish This Article under This Title (Old, million-dollar violins don't play better than the new models) without knowing anything about research design or inferential vs descriptive statistics. As a result, Ars Technica has shared with the public a piece of research that fails utterly to support the claim made in the title.

To wit:

The number of subjects in the study is: 21. This is too small a number of cases to merit computation of anything other than very simple descriptive statistics (e.g., "The number of people in the group is; their ages are," etc.). You may note that the authors made no attempt to generalize beyond these 21 individuals, and neither should you.

This is not to suggest that the investigator-authors did a fabulous job, either. For instance, I see no mention in the technical abstract (which I did download from the publisher's site) of 'inter-rater reliability,' which is a necessary computation in any research that asks people to rate and/or compare anything to anything else. Unless one has established, empirically, a reason to believe that, for each subject, there is consistency in ratings from one episode to another, one cannot tell whether or not one is receiving reliable reports - period. Moreover, the elaborate reporting of statistics like effect size, F, and p value may convey the appearance of probity. However, that appearance is highly misleading since the fundamental requirements for acceptable study design, implementation, and sample size have not been met.

All in all, this "research" does not merit coverage in your fine publication, and it certainly does not merit the conclusions drawn either by this story's author or by the original authors. 
 

In short, this kind of research is a classic case of fools rush in where angels fear to tread. There's a lot of rushing going on these days, so be careful to watch your step!

                                                                                               Stradivarius Violin

Behavioral Science, Empiricism, and Ingenuity

This morning's New York Times features a terrific article, Social Change's Age of Englightenment by David Bornstein. In the post, Bornstein shows how the growing realization that "we are not econs," combined with rigorous, disciplined collection/analysis/use of empirical data, can help us reframe intractible dilemmas in ways that allow breakthrough change.

Certainly, this matches my experience. I adopted the approach Bornstein describes when I worked in banking research. At some random point, it occurred to me that what we call 'money' is not a thing, but a symbol. Paper money, coins, and/or data flowing through an ACH system: none have inherent value. The value is found in the meaning we give them. They are no more or less than buckets of meaning. "But what is that meaning?" I asked myself. Although the field had no name at the time (late 1980's), it turned out that I had stumbled upon behavioral economics.

Bornstein does a terrific job of articulating what the approach entails, the tools we use, and the value that it can deliver. I'm grateful to him for his cogent description and compelling argument.