Re-read of the book
“How To Measure Anything (Finding the Value on ‘Intangibles’
in Business” (3rd edition) by Douglas W. Hubbard

Responses to Thomas Cagley’s re-read Saturday blog posting series.
I am posting my responses here, before a reply over there (chapter by chapter).

Part-1:  key concepts are what are measurements, why measure, what is risk, choosing what to measure (Chapters 1 – 7).

Part-2 (this post) is about the rest of the book, chapters 8 – 14 of the re-read.

Chapter 8:  The Transition: from What to Measure to How to Measure

The title of this chapter (The Transition: from What to Measure to How to Measure) is perfect for moving forward into part-2 of HTMA.

Hubbard summarizes what we can do to improve our measurements at the end of the chapter (pages 195 – 195) — (1) Work through the consequences, (2) Be iterative (yes! sounds familiar), (3) Consider multiple approaches, (4) What’s the really simple question that makes the rest of the measurements moot, and (5) Just do it.

My Notes from Chapter 8

  1. (p. 176-177) 6 questions to help determine the measurement methods
  2. (p. 178-179) 7 measurement instruments
  3. (p. 181) Decompose it (definition)
  4. (p. 183) Decomposition effect
  5. (p. 187) Some basic methods of observations
  6. (p. 190) Quick Glossary of (Measurement) Error
  7. (p. 193) A Few Types of Observation Biases
  8. (p. 194) Chose and Design the Instrument


Chapter 9: Sampling Reality How Observing Some Things Tells Us About All Things

There is a lot of information in this chapter.  Hubbard’s narrative discussing how to measure the number of fish in a lake (p. 214 – 215) helps me understand how this book lives up to its title.

My Notes from Chapter 9

  1. Mathless 90% CI, p. 211 Exhibit 9.4
  2. See relations in the data, p. 236  Examples of Correlated Data
  3. p. 242  The tw0 biggest mistakes in interpreting correlation.
    1. Correlation proves causation
    2. Correlation isn’t evidence of causation

Chapter 10:  Bayes:  Adding to What You Know Now

p. 247  “One of the key assumptions in most introduction-to-statistics course is that the only thing you ever knew about a population are the samples you are about to take.  In fact, this is virtually never true in real-life situations.”

p. 248 “Bayes’ theorem is simply a relationship of probabilities and “conditional” probabilities. …

p. 258 the Instinctive Bayesian Approach

p. 262 Exhibit 10.5 Confidence versus Information Emphasis

p. 264 Peter Tippett overcoming the “all things must be done” thinking that prevents measurements.

P. 276 “The Lessons of Bayes” (summary)

Chapter 11:  Preference and Attitudes:  The Softer Side of Measurement

The hypothetical utility curves helps with subjective trade-off evaluations (pages 300-301).

Example:  how does the performance of software team completing on-time at 99% with a 95% error-free rate compare with another software team completing on-time at 92% with a 99% error-free rate?  Check the organization’s utility curve for this trade-off.

My Notes

  1. Stated preferences versus Revealed preferences … Lean thinking
  2. This chapter has some good tips about designing surveys that can help measure and reduce uncertainty.
  3. p. 291 correlate subjective responses with objective measures (see Measuring Happiness)

Chapter 12:  The Ultimate Measurement Instrument:  Human Judges

Very good chapter notes!

Adding to (p. 325) “The Big Measurement Don’t – Above all else, don’t use a method that adds more error to the initial estimate.”, Hubbard warns us about using arbitrary scores (e.g., a scale of 1 – 5).

(p. 327) “I’ve always considered an arbitrary score to be a sort of measurement wannabe.” and then proceeds to list six reasons to support his statement.

Chapter 13:  New Measurement Instruments for Management

This chapter is about new ways of measuring using resources on the internet:  two books called out are

  1. Eric Siegel “Predictive Analytics:  The Power to Predict Who Will Click, Buy, Lie, or Die”
  2. Hubbard’s third book “Pulse:  The New Science of Harnessing Internet Buzz to Track Threats and Opportunities”

Pages 351 – 352, Hubbard summarizes four Subjective Assessment Methods to Prediction Markets, including what Hubbard discussed in this chapter, the Prediction Market.  The other three from previous chapters include (1) Calibration Training, (2) Lens Model, and (3) Rasch Model.

Thank you Thomas for selecting Commitment – Novel About Managing Project Risk by Olav Maassen, Chris Matts, and Chris Geary (Illustrator)  as the next re-read.  It will be a quick and fun read, and help any project leader.

My 90% calibration estimate to complete this re-read is 2 – 3 weeks, even though it is 216 pages (hard cover edition); the pages turn fast!

Chapter 14:  A Universal Measurement Method: Applied Information Economics

I like the summary of this book which comes from question #23 (Chapter 14)  in the HTMA Workbook, and I am quoting both the question and answer …

Summarize six points the author makes about the AIE philosophy.

  1. If it’s really that important, it’s something you can define.  If it’s something you think exists at all, it’s something you’ve already observed somehow.
  2. If it’s something important and something uncertain, you have a cost of being wrong and a chance of being wrong.
  3. You can quantify your current uncertainty with calibrated estimates.
  4. You can compute the value of information by knowing the “threshold” of the measurement where it begins to make a difference compared to you existing  uncertainty.
  5. Once you know what it is worth to measure something, you can put the measurement effort in context and decide on the effort it should take.
  6. Knowing just a few methods for random sampling, controlled experiments, Bayesian methods, or even merely improving on the judgements of experts can lead to a significant reduction in uncertainty.

Hubbard last paragraph in the HTMA book tells to how to start applying this knowledge (p. 385) “… and the practical cases described make you a little more skeptical about claims that something critical to your business cannot be measured”.

Last words about HTMA

Nice summary of HTMA Thomas!

This material, like running, takes more than just reading about it. It takes practice / training, and there are supplemental Excel worksheets online to study.

One of my favorite parts of HTMA is where Hubbard explains how to estimate the number of fish in a lake (p, 214 – 215).

The bridge: HTMA briefly discusses options and how “real” Options are over-used (p. 383-384). One of the themes in the next book up “Commitment…” is “real” Options. It will be interesting to compare notes.