This is primarily a course about prediction. In common speech, prediction involves using information that is available now to evaluate the likelihood of an uncertain event in the future. In epidemiology and biostatistics, “prediction” includes using available information not only to predict the future but also to estimate the probability of a state or condition that already exists but is difficult or expensive to measure by other means. In public health and clinical practice, diagnostic tests are used to estimate the probability of prevalent disease and risk prediction models are used to evaluate the likelihood of some future event. In this course, we will cover how to interpret the metrics used to describe the performance of diagnostic tests and risk prediction models, how to design research studies to evaluate tests and risk models, and how to use the results of tests and risk models to inform decision-making. Throughout, we assume that the information from tests and models guides decisions. Although the tests and models discussed are clinical and the decisions are often treatment decisions, the principles apply to any problem of prediction and decision-making under uncertainty.
The specific objectives of this course are to provide a basic understanding of:
- sensitivity, specificity, predictive value;
- likelihood ratios, ROC curves;
- inter-observer agreement, reliability, and measurement error;
- calibration plots, net benefit calculations, decision curves;
- logistic regression, and recursive partitioning (at an introductory level);
- special issues related to the evaluation of screening tests and programs; and
- quantifying treatment benefits and harms using the result of randomized trials
Designing Clinical Research (EPI 202). Exceptions may be made with the consent of the Course Director, space permitting. The course draws heavily upon clinical examples and may be more challenging for students without any clinical background. However, learning how to use clinical information to diagnose disease or predict outcomes and guide treatment decisions is an excellent way to introduce prediction in general.
Michael Kohn, MD, MPP
|Course Director Emeritus:||
Tom Newman, MD, MPH
|Faculty Section Leaders:||
Kerstin Kolodzie, MD, PhD, MAS
Martina Steurer, MD, MAS
Miriam Laker-Oketta, MD, MSc
|Teaching Assistant Section Leaders:||
Jonathan Amatruda, MD
Akansha Batra, MA
Griffin Collins, MD
Each week, new material is introduced via a recorded lecture and recommended readings. After beginning to study the lecture and reading, the class gathers for a large group discussion in which the lecture is briefly reviewed and students have the opportunity to pose questions to course faculty or prompt discussion on any aspect of the material. Homework, in the form of a problem set, is assigned each week. The goal of the homework is to reinforce the main points brought forth in lecture as well as to cover more detailed nuances found in the readings. The problem sets are discussed in detail with course faculty in the small group discussion sections that occur at the end of the weekly cycle.
Large Group Discussion
Content: Brief formal review of lecture followed by question and answer discussion. Recorded lecture should be viewed prior to this session. The video can played at 0.5 to 2.0x speed, depending upon your desire.
Time: Tuesdays, 8:45 to 10:15 AM, beginning September 22
Small Group Discussion
Content: Overview and discussion of lectures, and review of homework assignments. In the event that not all homework problems are discussed, a detailed answer key is always made available online shortly after the session.
Time: Thursdays, 1:15 to 2:45 PM, beginning September 17
Content: Course faculty are available to address questions on course content
Time: Wednesdays, 1:00 to 2:30 PM, beginning September 23
All course materials and handouts will be posted on the course's online syllabus.
Evidence-Based Diagnosis by T. Newman and M. Kohn with illustrations by Martin Steurer. Cambridge University Press. 2nd Edition. 2020. UCSF-affiliated students can download a free .pdf through the UCSF Library at the following link: http://ucsfcat.library.ucsf.edu/record=b2792723.
Some of our material can also be found (in abbreviated form)
in Designing Clinical Research, by Stephen B. Hulley, MD, MPH et al. Lippincott Williams & Wilkins. 4th Edition. 2013. Chapter 12 is particularly useful and is partly based on this course.
Stata Statistical Software (Stata Corporation, College Station, TX) is not required for any of the homework or exam problems, but some students find it useful for the unit on inter-rater agreement. Version 13 or higher is acceptable. A six-month student license for Stata/IC is the least expensive option that will be suitable to complete all course assignments, but Stata/SE is recommended for robust future use. The TICR Program has arranged for a sizeable discount for UCSF-affiliated personnel.
Grading is based equally on homework (including the problem-writing assignment, which counts as 1 homework) and a take-home final exam. Students will turn in 9 problem sets, a problem-writing assignment (due at 8:30 am on Tuesday 11/17), and a final exam (due at 8:30 am on Tuesday 12/8/2020). The problem sets are due at the beginning of each Thursday small group session. Except for the first Thursday (9/17), the small-group session will be devoted to reviewing the problem set that students have just turned in. On the first Thursday, we will play the "Kappa Game". You should try to watch the first lecture, read Chapters 1 and 5 of the textbook, and do the problem exercises prior to the small group session on 9/17.
Students not in full-year TICR Programs who satisfactorily pass all course requirements will, upon request, receive a Certificate of Course Completion.
This course is sponsored by the Training in Clinical Research (TICR) Program, and space is limited. Preference is given to UCSF-affiliated personnel. We regret that auditing in the classroom is not permitted, but most of the course materials (with the exception of videotapes, answer keys, examinations, and copyrighted documents) are freely available (without formal enrollment) on the course’s online syllabus. Many students can glean the majority of the course’s content from this free access, but, importantly, formal enrollment also provides access to faculty for questions and individual-level extension of the curriculum, a community of other engaged students for in-person real-time discussion, and personalized correction and feedback on homework and projects.
To enroll in this course, please fill out and submit the application below. Please see our fees page for cost information. The deadline for application is September 7, 2020. Only one application needs to be completed for all courses desired during the quarter.
The application is best completed using the latest version of Firefox, Chrome or Safari.
Information for how to pay;
please read before applying