Page 48 - The Real Work Of Data Science Turning Data Into Information, Better Decisions, And Stronger Organizations by Ron S. Kenett, Thomas C. Redman (z-lib.org)_Neat

8

When the Data Leaves Off

and Your Intuition Takes Over

It may seem trite to mention, but the power in data science lies not so much in uncovering
what the data reveals about itself but what it reveals beyond itself. In this chapter, we focus
on the process of moving from numbers to data, to information and insights (Kenett 2008),
what we call “generalizability” in the information quality framework to be introduced in
Chapter 13.
We recognize that some data is “soft,” based on a person’s experience, impressions, and
feelings. Soft data stands in contrast to “hard data,” particularly that which is digitized. And
although we often prefer hard data, data scientists must not discount soft data. It is often valid
and may well go far beyond the hard data. Good data scientists aim to combine the two. See
Appendix B for a full discussion of what we mean by hard data, soft data, and information.
Figure 8.1 summarizes the situation. We want to make valid inferences and predictions
from the data about bigger, more important areas of interest to decision‐makers. Briefly, we
recognize four distinct methods for doing so:

• The “laws of nature” refers to laws and models that allow one to extrapolate, under assumptions.
• “Statistical generalization” refers to making inference from a sample (of hard data) to a target
population.
• “Domain‐specific generalization” refers to applying domain knowledge, not fully supported
by the hard data, to other circumstances, such as the future or different populations.
• “Intuition” refers to people’s ability to reason from the data in ways that cannot be fully
explained. Science in general and data science in particular often discount intuition. But it
is undeniably true that some decision‐makers have unerring intuition. And at the least, it is
necessary because all decisions are made in the face of uncertainty.

To leave no doubt: data scientists should bring as much data, both hard and soft, and
combine all types of generalizations in the most powerful, transparent ways they can to help
decision‐makers!

The Real Work of Data Science: Turning Data into Information, Better Decisions, and Stronger Organizations,
First Edition. Ron S. Kenett and Thomas C. Redman.
© 2019 Ron S. Kenett and Thomas C. Redman. Published 2019 by John Wiley & Sons Ltd.
Companion website: www.wiley.com/go/kenett-redman/datascience