This is primarily a course about prediction. In common speech, prediction involves using information that is available now to evaluate the likelihood of an uncertain event in the future. In epidemiology and biostatistics,“prediction” includes using available information not only to predict the future but also to estimate the probability of a condition that already exists but is difficult or expensive to establish definitively. In public health and clinical practice, diagnostic tests are used to estimate the probability of prevalent disease and risk prediction models are used to evaluate the likelihood of incident outcomes. In this course, we will cover how to
- interpret the metrics used to describe the performance of diagnostic tests and risk prediction models;
- design research studies to evaluate tests and risk models;
- critically appraise studies of diagnostic tests, and
- use the results of tests and risk models to inform decision-making.
Throughout, we assume that the information from tests and models guides decisions. Although the tests and models discussed are clinical and the decisions are often treatment decisions,
the principles apply to any problem of prediction and decision-making under uncertainty.
The specific objectives of this course are to provide a basic understanding of:
- sensitivity, specificity, predictive value;
- likelihood ratios, ROC curves;
- inter-observer agreement, reliability, and measurement error;
- calibration plots, net benefit calculations, decision curves;
- multivariable risk models, both development and evaluation;
- special issues related to the evaluation of screening tests and programs; and
- quantifying treatment benefits and harms using the results of randomized trials and observational studies.
Designing Clinical Research (EPI 202). Exceptions may be made with the consent of the Course Director, space permitting. The course draws heavily upon clinical examples and may be more challenging for students without any clinical background. However, learning how to use clinical information to diagnose disease or predict outcomes and guide treatment decisions is an excellent way to introduce prediction in general.
Michael Kohn, MD, MPP
|Course Director Emeritus/Current Co-Director:||
Tom Newman, MD, MPH
|Faculty Section Leaders:||
Alexis Beatty MD, MAS
Martina Steurer, MD, MAS
Miriam Laker-Oketta, MD, MSc
|Teaching Assistant Section Leaders:||
Vidya Eswaran, MD
Anita Hargrave, MD
Nicole Rodriguez, MS
Nasim Sobhani, MD
Each week, new material is introduced via a recorded lecture and recommended readings. After beginning to study the lecture and reading, the class gathers for a large group discussion in which the lecture is briefly reviewed and students have the opportunity to pose questions to course faculty or prompt discussion on any aspect of the material. Homework, in the form of a problem set, is assigned each week. The goal of the homework is to reinforce the main points brought forth in lecture as well as to cover more detailed nuances found in the readings. The problem sets are discussed in detail with course faculty in the small group discussion sections that occur at the end of the weekly cycle.
Large Group Discussion
Content: Brief formal review of lecture followed by question-and-answer. Recorded lecture should be viewed prior to this session. The video can played at 0.5 to 2.0x speed, depending upon your desire.
Time: Tuesdays, 8:45 to 10:15 AM, beginning September 21
Small Group Discussion
Content: Overview and discussion of lectures, and review of homework assignments. A detailed answer key is always made available online shortly after the session.
Time: Thursdays, 1:15 to 2:45 PM, beginning September 16
Content: Course faculty are available to address questions on course content
Time: Wednesdays, 1:00 to 2:30 PM, beginning September 22
All course materials and handouts will be posted on the course's online syllabus.
Evidence-Based Diagnosis by T. Newman and M. Kohn with illustrations by Martina Steurer. Cambridge University Press. 2nd Edition. 2020. UCSF-affiliated students can download a free .pdf through the UCSF Library at the following link: http://ucsfcat.library.ucsf.edu/record=b2792723.
Some of our material can also be found (in abbreviated form)
in Designing Clinical Research, by Stephen B. Hulley, MD, MPH et al. Lippincott Williams & Wilkins. 4th Edition. 2013. Chapter 12 is particularly useful and is partly based on this course.
Stata Statistical Software (Stata Corporation, College Station, TX) is not required for any of the homework or exam problems, but some students find it useful for the unit on inter-rater agreement. Version 13 or higher is acceptable. A six-month student license for Stata/IC is the least expensive option that will be suitable to complete all course assignments, but Stata/SE is recommended for robust future use. The TICR Program has arranged for a sizeable discount for UCSF-affiliated personnel.
Grading is based equally on homework (including the problem-writing assignment, which counts as 1 homework) and a take-home final exam. Students will turn in 9 problem sets, a problem-writing assignment, and a final exam. The problem sets are due at the beginning of each Thursday small group session. Except for the first Thursday (9/16), the small-group session will be devoted to reviewing the problem set that students have just turned in. On the first Thursday, we will play the "Kappa Game". You should watch the first lecture, read Chapters 1 and 5 of the textbook, and do Problem Set 0 (not to be turned in) prior to the small group session on 9/16.
Students not in full-year TICR Programs who satisfactorily pass all course requirements will, upon request, receive a Certificate of Course Completion.
This course is sponsored by the Training in Clinical Research (TICR) Program, and space is limited. Preference is given to UCSF-affiliated personnel. We regret that auditing in the classroom is not permitted, but most of the course materials (with the exception of videos, answer keys, examinations, and copyrighted documents) are freely available (without formal enrollment) on the course’s online syllabus. Many students can get most the majority of the course’s content from this free access, but, formal enrollment provides access to faculty for questions, a community of other engaged students for in-person real-time discussion, and personalized correction and feedback on homework and projects.
To enroll in this course, please fill out and submit the application below. Please see our fees page for cost information. The deadline for application is September 3, 2021. Only one application needs to be completed for all courses desired during the quarter.
The application is best completed using the latest version of Firefox, Chrome or Safari.
Information for how to pay;
please read before applying