Page 75 - The Real Work Of Data Science Turning Data Into Information, Better Decisions, And Stronger Organizations by Ron S. Kenett, Thomas C. Redman (z-lib.org)_Neat

Evaluating Data Science Outputs More Formally 65

1. Answer the following: Who are this organization’s most important customers and sup-
pliers? What are its most important products and services?
2. Select an important goal for this organization. It can be reducing costs, improving quality,
or gaining new customers. This step defines the g and U components in the InfoQ equation.

Step 2: The Data
List various data sources that are available to help decision‐makers pursue that goal. In eval-
uating data sources, focus on data quality and data clarity. Data quality reflects the extent to
which the data can be trusted, and data clarity represents the way data elements are defined
and collected by various parts of the organization. This step specifies the X component of
InfoQ.
Step 3: The Analysis
Identify several approaches for analyzing the data in order to help the organization achieve its
goal. In this step, identify and list alternative methods of analysis, f , f , …, f .
1 2 p
Step 4: Assessment
Assess the data and the potential analysis on eight InfoQ “dimensions” using a 1–5 score
where 1 means “very poorly” and 5 “very well.”
1. Data resolution. When the data is on the right level of granularity, the measurement scale
is appropriate, and the level of aggregation is appropriate, score a “5.”
2. Data structure. When there are important gaps in the data coverage, score a “1.”
3. Data integration. A “5” corresponds to integration into a seamless whole.
4. Temporal relevance. When the data is timely with respect to the goal, score a “5.”
5. Generalizability. When what we learn can be generalized to many other circumstances,
score a “5.”
6. Chronology of data and goal. When the analysis and recommendations can be completed
in a timely fashion from a decision‐making perspective, score a “5.”
7. Operationalization. If the analyses are unlikely to lead to concrete actions that provide
business benefit, score a “1.”
8. Communication. If the “who” (needs the information), “what,” “when,” “why,” and “how”
are clear, score a “5.”

Note: an application for recording InfoQ scores, which also allows for a range of values
reflecting uncertainty in the score, is available for download from the Wiley website of
Kenett and Shmueli (2016a). The application requires installation of the JMP software
and provides an overall InfoQ score based on the geometric mean of the individual
dimension scores.
Phase II: Teamwork
Form teams of three or four participants.

1. Share your case study with the team and engage in open discussion.
2. Choose the case study that your team will present in the group session.
3. Prepare your case study.