Design of Observational Studies

Rosenbaum’s Design of Observational Studies doesn’t skip steps, that’s why the book is so hefty… and so useful.

 

For someone who has been around causal inference talk but not completely immersed in the field, this book helps fill in the gaps, step by step, and shows how ideas like matching, sensitivity analysis, and instrumental variables were developed and all fit together to make the most out of observational data. The content is accessibly explained, building on intuition by starting with what makes an experiment so useful statistically, and then explaining what makes the observational study different, opening up the doors to potential pitfalls that now must be addressed, ideally in the design phase. If a reader with some statistical background is starting from scratch, with no prior knowledge of causal inference, the building blocks are still present. They just may require a slower read at the beginning of the book. 

 

Rosenbaum provides an accurate roadmap for the book at the end of Chapter 1 and has reminders throughout the text about what sections can be skipped, like those with extended proofs or other mathematical details, and what sections would further the current discussion. Because the book is modularized for easy skipping around, there is some built-in redundancy to anticipate readers needing a refresher at any moment. These short portions of review have the added benefit of clarifying a concept that didn’t quite sink in upon the first read earlier on in the text. Rosenbaum has a way of saying the same thing in multiple ways to try to triangulate understanding. 

 

The book is focused on the conceptual, grounded in real-world examples, but implementation is not neglected. Although there is minimal R code in the book itself, the accompanying R package DOS2, new to the second edition, has examples (both data and code) in the documentation that dovetail with the examples described in the book. When other R packages are more appropriate for analysis, these are referenced as well. There is also an accompanying interactive Shiny app that allows the user to gain intuition about sensitivity analysis by toggling different choices of test statistics and sensitivity parameters for an example from the book. It would be nice to have R scripts used in each example in a separate location rather than spread throughout the documentation, but the book does provide guidance to what functions are used for different examples to give the reader a sense of where to start looking.

 

Avoiding overemphasis on the mathematical details, the book does not contain exercises, so a faculty member looking to use this book for a class would need to craft their own problem sets (another place where a little extra R code organization would be helpful). However, this book is readily usable for self-study, a graduate seminar, or use by a practitioner getting their bearings in the field. The book is filled with references to both foundational work and cutting-edge approaches. As part of the backmatter, these many references are boiled down to a shorter list recommended for a course on the design of observational studies. For the practitioner, the end of the book provides a Q&A section, “Solutions to Common Problems,” that gives high-level advice for situations faced by the reader and points them to the relevant portions of the book to learn more about the details. 

 

Despite its heft (with four new chapters and multiple new sections sprinkled throughout since the first edition), the book was an easy read. Rosenbaum provides practical advice, delivered in a conversational tone, and steers clear of the weeds that might bog a reader down in the main text. I learned a lot, from little tricks (“In general, if you want a two-sided P-value, compute both one-sided P-values, double the smaller one, and take the minimum of this value and 1. This approach views the two-sided P-value as a correction for testing twice.”) to bigger picture strategies for crafting statistical arguments about the plausibility of causal effects even without a gold standard randomized experiment.

 

Importantly, the book provides a philosophy for approaching questions using observational data with two main pillars: (1) matching to remove observable selection bias and (2) sensitivity analysis as a way to account for the part of bias we can never convince ourselves doesn’t exist in observational data.

---

 

Sara Stoudt (https://sastoudt.github.io/) is an assistant professor in the Department of Mathematics at Bucknell University. She is interested in applied statistics and the pedagogy of writing in the STEM fields.