Recent Workshops in Biostatistics

Prioritizing likely causal genes from GWAS data is a fundamental problem. There are many methods for GWAS gene prioritization, including methods that map candidate SNPs to their target genes, and methods that leverage patterns of enrichment from across the genome. In this talk, I will introduce a new method for leveraging genome-wide patterns of enrichment to prioritize genes at GWAS loci, incorporating information about genes from many sources. I will then discuss the problem of benchmarking gene prioritization methods, and I will describe a large-scale analysis to benchmark many different methods and combinations of methods on data from the UK Biobank. Our analyses show that the highest confidence can be achieved by combining multiple lines of evidence, and I will conclude by giving examples of genes prioritized in this way.

Access Zoom recording. 

Single cell sequencing experiments almost exclusively analyze cells' "gene expression". Yet, more than 95% of human genes are alternatively spliced or subject to regulated RNA processing that can result in isoforms of the same "gene" with opposite functions. Despite the need to quantify RNA variants of the same gene, methods to do so are lacking in the field. Dr. Salzman will discuss two new statistical methods to identify cell type-specific RNA regulation: 1) generalization of a metric on which the splicing field has long relied: “Percent Spliced In” (PSI) called the "Splicing Z score" (SpliZ); and 2) a new metric to detect alternative RNA processing. Both methods can be applied to any RNAseq data, however our focus is on using them to study single cell RNA-seq. Dr. Salzman will discuss analysis of single cell RNAseq data including findings from analysis of > 100K carefully curated single cells from 12 human tissues and spermatogenic development.

Access Zoom recording. 

The gut microbiome plays an important role in maintenance of human health. High-throughput shotgun metagenomic sequencing of a large set of samples provides an important tool to interrogate the gut microbiome. Besides providing footprints of taxonomic community composition and genes, these data can be further explored to study the bacterial growth dynamics and metabolic potentials via generation of small molecules and secondary metabolites. In this talk, Dr. Lee will present several computational and statistical methods for estimating the bacterial growth rate for metagenome-assembled genomes (MAGs) and for predicting all biosynthetic gene clusters (BGCs) in bacterial genomes. The key statistical and computational tools used include optimal permutation recovery based on low-rank matrix projection and improved LSTM deep learning methods to improve prediction of BGCs. He will demonstrate the application of these methods using several ongoing microbiome studies of inflammatory bowel disease at the University of Pennsylvania.

Access Zoom recording. 

Genomics can be used to quantify gene expression and characterize gene regulatory elements in human tissues, including how genetic mutations alter gene regulation in disease. Single-cell and bulk tissue experiments have complementary strengths and weaknesses, and alone neither strategy can fully capture regulatory elements across the diversity of cells in complex tissues. To solve this problem, we developed CellWalker, a method that integrates single-cell open chromatin (scATAC-seq) data with gene expression (RNA-seq) and epigenetic data using a network model. We demonstrate the model’s robustness to sparse annotations and measurement noise using simulations and combined RNA-seq and ATAC-seq in individual cells. Applying CellWalker to the developing brain and to heart disease, we identify cells transitioning between states, resolve regulatory elements to cell types, and map disease mutations to specific kinds of cells. Our modeling approach has been implemented in an R package called CellWalkR.

Access Zoom recording. 

Increase in the number of copies of tumor promoting (onco-) genes is a hallmark of many cancers, and cancers with copy number amplifications are often associated with poor outcomes. Despite their importance, the mechanisms causing these amplifications are incompletely understood. In this talk, we describe our recent results suggesting that a large faction of amplification is due to formation of extrachromosomal DNA (ecDNA). EcDNA play a critical role in tumor heterogeneity, accelerated cancer evolution, and drug resistance through their unique mechanism of non-chromosomal inheritance. While predominant, ecDNA are not the only mechanism to cause amplification. We also describe recent algorithmic methods required to distinguish ecDNA from other mechanisms including Breakage Fusion Bridge formation, Chromothripsis, and simpler events such as tandem duplications and translocations. The talk is a mix of published and unpublished work, largely in collaboration with Paul Mischel's lab at UCSD. EcDNA was recently recognized as one of the grand challenges of cancer research by Cancer Research UK and the National Cancer Institute.

Access Zoom recording. 

As machine learning methods become embedded in society, it has become clear that the data used, objectives selected, and questions we ask are all critical. My work looks at data and machine learning from a public health and equity lens. First, this motivates the design and development of data mining and machine learning methods to address challenges related to data and goals of public health, such as generating better hyper-local features to represent environmental attributes addressing challenges of sparsity, irregularity and representativeness of data. Second, principles of community and equity inspire innovations in machine learning. In this realm my work has leveraged causal models and machine learning to address realistic challenges of data collection and model use across environments, such as domain adaptation that improves prediction in under-represented population sub-groups by leveraging invariant information across groups when possible, and developing models to specifically incorporate structural factors to better account for and address sources of bias and disparities. A focus on public health, which is concerned with the individual, collective, and environmental factors that affect the health of human populations provides a principled approach spanning data, algorithms and questions to both mitigate bias and proactively design inclusive innovations.

Access Zoom recording. 

The history of computers collecting and analyzing clinical data goes back to the 1950s, beginning with Alan Turing’s speculations and continuing with Morris Collen’s pioneering collection of multiphasic health checkup data at Kaiser, and Robert Ledley and Lee Lusted’s outline of how one could use symbolic logic and decision analysis to help understand physicians’ reasoning. The 1960s to 1980s saw development of new models for clinical reasoning based on rules, matching of prototypical patterns, explicit modeling of human decision making, sophisticated methods of probabilistic reasoning, etc., and applications of these ideas to clinical decision support. Since the 1990s, large collections of experiential data from electronic health records have become more widely available, and consequently we have turned to machine learning methods to build classification and prediction models from such data. The next challenges, in my view, focus on how one can combine biomedical knowledge with clinical data collections of heterogeneous data types to learn models that are accurate, explainable, fair, and clinically useful.

Access Zoom recording. 

Digital Healthcare is a growing area of importance in modern healthcare due to its potential in helping individuals improve their behaviors so as to better manage chronic health challenges such as hypertension, mental health, cancer and so on. Digital apps and wearables, observe the user's state via sensors/self-report, deliver treatment actions (reminders, motivational messages, suggestions, social outreach,...) and observe rewards repeatedly on the user across time. This area is seeing increasing interest by reinforcement learning (RL) researchers with the goal of including in the digital app/wearable an RL algorithm that "personalizes" the treatments to the user. But after RL is run on a number of users, how do we know whether the RL algorithm actually personalized the sequential treatments to the user? In this talk, we report on our first efforts to address this question after our RL algorithm was deployed on individuals with hypertension.

Access Zoom recording.

The chromosomes of the human genome are organized in three-dimensions by compartmentalizing the cell nucleus and different genomic loci also interact with each other. However, the principles underlying such nuclear genome organization and its functional impact remain poorly understood. In this talk, I will introduce some of our recent work in developing machine learning methods by utilizing whole-genome mapping data to study the higher-order genome organization. In particular, I will highlight a new method based on hypergraph representation learning to probe the 3D genome structures in single cells. Our methods reveal the spatial localization of chromosome regions and exploit chromatin interactome patterns within the cell nucleus in different cellular conditions and at different scales. We hope that these algorithms will provide new insights into the structure and function of nuclear organization in health and disease.

Access Zoom recording. 

Germline mutation is the source of all heritable differences and therefore of fundamental importance. In mammals, we know from longstanding analyses of phylogenetic patterns on the X-chromosome and autosomes, and from recent studies of human pedigrees, that most mutations come from fathers, and more are transmitted from older parents than from younger ones. The textbook view is that these patterns reflect replication errors, as there are both more mutations and more cell divisions in the male than in the female germline. I will present multiple lines of evidence that call this view into question. I will argue instead that current data are best explained by a much larger role of DNA damage in the genesis of germline mutations than previously appreciated, and draw out implications for why mutation rates depend on sex and age and how they evolve over time.

Glioblastoma are notoriously aggressive, malignant primary brain tumors that have variable response to treatment. This presentation will focus on the integrative role of 1) biological sex-differences, 2) heterogeneity in drug-delivery and 3) intra-tumoral molecular diversity (revealed by radiomics) in capturing and predicting this variable response to treatment. Specifically, I will highlight burgeoning insights into sex differences in tumor incidence, outcomes, propensity and response to therapy. I will further, quantify the degree to which heterogeneity in drug delivery, even for drugs that are able to bypass the blood-brain barrier, contributes to differences in treatment response. Lastly, I will propose an integrative role for spatially resolved MRI-based radiomics models to reveal the intra-tumoral biological heterogeneity that can be used to guide treatment targeting and management.

Access zoom recording.

Data is the foundation on which statistical modeling rests. To provide a better foundation for COVID-19 tracking and forecasting, the Delphi group launched an effort called COVIDcast, which has many parts: 1. Unique relationships with partners in tech and healthcare granting us access to real-time data on pandemic activity. 2. Code and infrastructure to build COVID-19 indicators, continuously-updated and geographically comprehensive. 3. A historical database of all indicators, including revision tracking, currently with hundreds of millions of observations. 4. A public API serving new indicators daily (and R and Python packages for client support). 5. Interactive maps and graphics to display our indicators. 6. Forecasting and modeling work building on the indicators. In this talk, I'll summarize the various parts, and highlight some interesting findings so far. I'll also describe ways you can get involved yourself, access the data we've collected, and leverage the tools we've built.

Access Zoom recording. 

The year 2020 has brought into focus a second pandemic of social injustice and systemic bias with the disproportionate deaths observed for minority patients infected with COVID. As we observe an increase in development and adoption of AI for medical care, we note variable performance of the models when tested on previously unseen datasets, and also bias when the outcome proxies such as healthcare costs are utilized. Despite progressive maturity in AI development with increased availability of large open source datasets and regulatory guidelines, operationalizing fairness is difficult and remains largely unexplored. In this talk, we review the background/context for FAIR and UNFAIR sequelae of AI algorithms in healthcare, describe practical approaches to FAIR Medical AI, and issue a grand challenge with open/unanswered questions.

Access Zoom recording. 

Five decades have passed in the evolution of Artificial Intelligence in Medicine (AIM), a field that has evolved substantially while tracking the corresponding changes in computer science, hardware technology, communications, and biomedicine. Emerging from medical schools and computer science departments in its early years, the AIM field is now more visible and influential than ever before, paralleling the enthusiasm and accomplishments of AI more generally. This talk will briefly summarize some of AIM history, providing an update on the status of the field as we enter our second half-century. My remarks on this subject will emphasize the role that Stanford played in the emergence of the field. They will also offer the perspective of an informatics journal editor-in-chief who has seen many state-of-the-art AIM papers and thereby recognizes the tension between applying existing methods to new problems and developing new science that advances the field in a generalizable way. In addition, the inherent complexity of medicine and of clinical care necessitates that we address not only decision-making performance but also issues of usability, workflow, transparency, safety, and the pursuit of persuasive results from formal clinical trials. These requirements contribute to an ongoing investigative agenda that means fundamental AIM research will continue to be crucial and will define our accomplishments in the decades ahead.

Access Zoom recording. Access presentation slides (PDF). 

DNA and RNA sequencing data analysis often begins with aligning sequencing reads to a reference genome, with the reference represented as a linear string of bases. But linearity leads to reference bias, a tendency to miss or misreport alignments containing non-reference alleles, which can confound downstream statistical and biological results. This is a major concern in human genomics; we do not want diagnostics and therapeutics to be differentially effective depending on a patient's genetic background. Meanwhile, recent Bioinformatics advances allow us to index and align sequencing reads to references that include many population variants. I will describe this journey from the early days of efficient genome indexing, continuing through graph-shaped references and references that include many genomes. I will emphasize recent results, including results from my group and collaborators showing how to optimize simple and complex pan-genome representations for effective avoidance of reference bias. Much of this work is collaborative with Travis Gagie, Christina Boucher, Alan Kuhnle and others.

Access Zoom recording. 

COVID-19 is an emerging respiratory infectious disease that has become a pandemic. In this talk, I will first provide a historical overview of the epidemic in Wuhan. I will then provide the analysis results of 32,000 lab-confirmed COVID-19 cases in Wuhan to estimate the transmission rates using Poisson Partial Differential Equation based transmission dynamic models. This model is also used to evaluate the effects of different public health interventions on controlling the COVID-19 outbreak, such as social distancing, isolation and quarantine. I will present the results on the epidemiological characteristics of the cases. The results show that multi-faceted intervention measures successfully controlled the outbreak in Wuhan. I will next present transmission regression models for estimating transmission rates in USA and other countries, as well as factors including intervention effects using social distancing, test-trace-isolate strategies that affect transmission rates. I will discuss estimation of the proportion of undetected cases, including asymptomatic, pre-symptomatic cases and mildly symptomatic cases, the chances of resurgence in different scenarios, prevalence, and the factors that affect transmissions. I will also present the US county-level analysis to study the demographic, social-economic, and comorbidity factors that are associated with COVID-19 case and death rates. I will also present the analysis results of >500,000 participants of the HowWeFeel project on health outcomes and behaviors in US, and discuss the factors associated with infection, behavior, and vaccine hesitancy. I will provide several takeaways and discuss priorities.

Access Zoom recording. 

Via a confluence of genomic technology and computational developments the possibility of network inference methods that automatically learn large comprehensive models of cellular regulation is closer than ever. This talk will focus on enumerating the elements of computational strategies that, when coupled to appropriate experimental designs, can lead to accurate large-scale models of chromatin-state and transcriptional regulatory structure and dynamics. We highlight four research questions that require further investigation in order to make progress in network inference: using overall constraints on network structure like sparsity, use of informative priors and data integration to constrain individual model parameters, estimation of latent regulatory factor activity under varying cell conditions, and new methods for learning and modeling regulatory factor interactions. I will contrast two recent studies from the lab that focus on inference from single-cell and spatial transcriptomics aimed at healthy and diseased brain and spinal tissues.

Access Zoom recording. 

Single-cell RNA-seq assays are being increasingly applied in complex study designs, which involve measurements of many samples, commonly spanning multiple individuals, conditions, or tissue compartments. Joint analysis of such extensive, and often heterogeneous, sample collections requires a way of identifying and tracking recurrent cell subpopulations across the entire collection, and an effective way of exploring contrasts between samples. We develop comparative approaches for analysis of case-control study designs, commonly used to study the impact of disease or a drug on a particular tissue. The analysis starts by establishing probabilistic mapping – a joint graph – connecting all cells within the collection. The graph can then be used to propagate information between samples and to identify cell communities that show consistent grouping across broad subsets of the collected samples. The contrast between conditions is then formulated in terms of i) compositional shifts between different cell populations, ii) transcriptional shifts within the distinct cell populations, and iii) within-group expression state variability. The compositional data analysis techniques are applied in the context of cell types hierarchies to suggest most parsimonious explanation of the changes. The differential expression analysis is used to identify most affected cell types, and characterize the likely functional interpretation of the cell type-specific changes. We illustrate the application of these methods in the context of different studies of human disease, including cancer and diseases affecting the brain.

Access Zoom recording. 

Single-cell RNA-Seq (scRNA-seq) is the most widely used high-throughput technology to measure genome-wide gene expression at the single-cell level. However, single-cell data present unique challenges that have required the development of specialized methods and software infrastructure to successfully derive biological insights. Compared to bulk RNA-seq, there is an increased scale of the number of observations (or cells) that are measured and there is increased sparsity of the data, or fraction of observed zeros. Furthermore, as single-cell technologies mature, the increasing complexity and volume of data require fundamental changes in data access, management, and infrastructure alongside specialized methods to facilitate scalable analyses. I will discuss some challenges in the analysis of scRNA-seq data and present some solutions that we have made towards addressing these challenges.

Access Zoom recording. 

Quantification of gene expression using single cell RNA-sequencing of brain tissues, can be a critical step in the understanding of cell development and differences between cells sampled from case and control subjects. We describe statistical challenges encountered analyzing expression of brain cells in the context of two projects. First, over-correction has been one of the main concerns in employing various data integration methods that risk removing the biological distinctions, which is harmful for cell type identification. Here, we present a simple yet surprisingly effective transfer learning model named cFIT for removing batch effects across experiments, technologies, subjects, and even species. Second, gene co-expression networks yield critical insights into biological processes, and single-cell RNA sequencing provides an opportunity to target inquiries at the cellular level. However, due to the sparsity and heterogeneity of transcript counts, it is challenging to construct accurate gene co-expression networks. We develop an alternative approach that estimates cell-specific networks for each single cell. We use this method to identify differential network genes in a comparison of cells from brains of individuals with autism spectrum disorder and those without.

Access Zoom recording. 

As the COVID-19 outbreak evolves, accurate forecasting continues to play an extremely important role in informing policy decisions. In this paper, we present our continuous curation of a large data repository containing COVID-19 information from a range of sources. We use this data to develop predictions and corresponding prediction intervals for the short-term trajectory of COVID-19 cumulative death counts at the county-level in the United States up to two weeks ahead. Using data from January 22 to June 20, 2020, we develop and combine multiple forecasts using ensembling techniques, resulting in an ensemble we refer to as Combined Linear and Exponential Predictors (CLEP). Our individual predictors include county-specific exponential and linear predictors, a shared exponential predictor that pools data together across counties, an expanded shared exponential predictor that uses data from neighboring counties, and a demographics-based shared exponential predictor. We use prediction errors from the past five days to assess the uncertainty of our death predictions, resulting in generally-applicable prediction intervals, Maximum (absolute) Error Prediction Intervals (MEPI). MEPI achieves a coverage rate of more than 94% when averaged across counties for predicting cumulative recorded death counts two weeks in the future. Our forecasts are currently being used by the non-profit organization, Response4Life, to determine the medical supply need for individual hospitals and have directly contributed to the distribution of medical supplies across the country. We hope that our forecasts and data repository at https://covidseverity.com can help guide necessary county-specific decision-making and help counties prepare for their continued fight against COVID-19.

RanElections rely on people, hardware, and software, all of which are fallible and subject to manipulation. Well resourced nation-states continue to attack U.S. elections and domestic election fraud is not unheard of. Voting equipment is built by private vendors--some foreign, but all using foreign parts. Many states even oursource election reporting to foreign firms. How can we conduct and check elections in a way that provides evidence that the reported winners really won--despite malfunctions and malfeasance? Evidence-based elections require voter-verified (generally, hand-marked) paper ballots kept demonstrably secure throughout the canvass and manual audits of election results against the trustworthy paper trail. Hand-marked paper ballots are far more trustworthy than machine-marked ballots for a variety of reasons. Two kinds of audits are required to provide affirmative evidence that outcomes are correct: _compliance audits_ to establish whether the paper trail is complete and trustworthy, and _risk-limiting audits_ (RLAs). RLAs test the hypothesis that an accurate manual tabulation of the votes would find that one or more reported winners did not win. To reject that hypothesis means there is convincing evidence that a full hand tally would confirm the reported results. For a broad variety of social choice functions, including plurality, multi-winner plurality, supermajority, proportional representation rules such as D'Hondt, Borda count, approval voting, and instant-runoff voting (aka ranked-choice voting), the hypothesis that one or more outcomes is wrong can be reduced to the hypothesis that the means of one or more lists of nonnegative numbers is not greater than 1/2. Martingale methods for testing such nonparametric hypotheses sequentially are especially practical. RLAs are in law in several states and have been piloted in more than a dozen; there have been roughly 60 pilots in jurisdictions of all sizes, including roughly 10 audits of statewide contests. Open-source software to support RLAs is available.

The scientific needs and computational limitations of the Twentieth Century fashioned classical statistical methodology. Both the needs and limitations have changed in the Twenty-First, and so has the methodology. Large-scale prediction algorithms - neural nets, deep learning, boosting, support vector machines, random forests - have achieved star status in the popular press. They are recognizable as heirs to the regression tradition, but ones carried out at enormous scale and on titanic data sets. How do these algorithms compare with standard regression techniques such as Ordinary Least Squares or logistic regression? Several key discrepancies will be examined, centering on the differences between prediction and estimation or prediction and attribution (that is, significance testing). Most of the discussion is carried out through small numerical examples. The talk does not assume familiarity with prediction algorithms.

Mapping gene-enhancer regulatory relationships is key to unraveling molecular disease mechanisms based on GWAS associations in non-coding regions. This problem is notoriously challenging: there is a many-to-many mapping between genes and enhancers, and enhancers can be located far from their target genes. Recently developed CRISPR regulatory screens (CRSs) based on single cell RNA-seq (scRNA-seq) are a promising high-throughput experimental approach to this problem. They operate by infecting a population of cells with thousands of CRISPR guide RNAs (gRNAs), each targeting an enhancer. Each cell receives a random combination of CRISPR gRNAs, which suppress the action of their corresponding enhancers. The gRNAs and whole transcriptome in each cell are then recovered through scRNA-seq. CRSs provide more direct evidence of regulation than existing methods based on epigenetic data or even chromatin conformation. However, the analysis of these screens presents significant statistical challenges, some inherited from scRNA-seq analysis (modeling single cell gene expression) and some unique to CRISPR perturbation screens (the confounding effect of sequencing depth). In this talk, I will first give some background on single cell CRISPR screen technology. I will then present the first genome-wide single cell CRS dataset (Gasperini et al. 2019) and discuss challenges that arose in its initial analysis. Finally, I will present a novel methodology for the analysis of this data based on the conditional randomization test. The key idea is to base inference on the randomness in the assortment of gRNAs among cells rather than on the randomness in single cell gene expression, since the former is easier to model than the latter.

Despite the importance of human health, we do not fundamentally understand what it means to be healthy. Health is unlike many recent machine learning success stories - e.g., games or driving - because there are no agreed-upon, well-defined objectives. In this talk, Dr. Marzyeh Ghassemi will discuss the role of machine learning in health, argue that the demand for model interpretability is dangerous, and explain why models used in health settings must also be "healthy". She will focus on a progression of work that encompasses prediction, time series analysis, and representation learning.

The coronavirus will likely kill thousands of Americans. But what if I told you about a serious threat to American national security. This emergency comes from climate change and air pollution. To help address this threat, we have developed an artificial neural network model that uses on-the-ground air-monitoring data and satellite-based measurements to estimate daily pollution levels dividing the country into 1-square-kilometer zones across the continental U.S. We have paired this information with health data contained in Medicare claims records from the last 12 years, which includes 97% of the population ages 65 or older. We also developed statistical methods for causal inference and computational efficient algorithms for the analysis of over 550 million health records. The result? This data science platform is telling us that federal limits on the nation’s most widespread air pollutants are not stringent enough. Our research shows that short- and long-term exposure to air pollution is killing thousands of senior citizens each year. Our research shows the critical new role of data science in public health and the associated methodological challenges. For example, with enormous amounts of data, the threat of unmeasured confounding bias is amplified, and causality is even harder to assess with observational studies. We will discuss these and other challenges.

Machine learning advances are opening new routes to more precise healthcare, from the discovery of disease subtypes for stratified interventions to the development of personalised interactions supporting self-care between clinic visits. This offers an exciting opportunity for machine learning techniques to impact healthcare in a meaningful way. In this workshop, I will present recent work on probabilistic graphical modelling to enable a more personalised approach to healthcare. The underlying motivation of these methods is to understand patient heterogeneity in order to provide more personalised treatment and intervention strategies. An important element of developing these models is collaboration with domain experts such as social scientists who have a deep understanding of the user perspective of these algorithms. We will use motivating examples from mental healthcare and asthma and allergic diseases.

In this talk, I will discuss three real-world Health applications of Machine Learning research, the progress that we have made in deployment to hospitals or directly to individuals, and where we hope to be heading next. In the first part, I will discuss Sepsis Watch, our Sepsis prediction system that has been deployed to the emergency departments of Duke University hospitals. This system performs prediction for incoming patients through a combination of Gaussian Processes, which estimate patient features in continuous time from uneven measurements, and Recurrent Neural Networks. Next I discuss Graph-coupled HMMs, work that we have done making individual-level predictions of disease spread in a social network in influenza, and how this might affect prediction abilities in other diseases, such as Coronavirus. Lastly, I will discuss the iOS app developed to record data on people with Multiple Sclerosis outside of a clinic environment, what collected data and basic analyses imply for our ability to do symptom and subpopulation prediction in this setting, and where we are headed in the future.

Every year, there are more than 4 million referrals made to child protection agencies across the US. The practice of screening calls is left to each jurisdiction to follow local practices and policies, potentially leading to large variation in the way in which referrals are treated across the country. While increasing access to linked administrative data is available, it is difficult for workers to make systematic use of historical information about all the children and adults on a single referral call. Jurisdictions around the country are thus increasingly turning to predictive modeling approaches to help distill this rich information. The end result is typically a single risk score reflecting the likelihood of a near-term adverse event. Yet the use of predictive analytics in the area of child welfare remains highly contentious. There is concern that some communities—such as those in poverty or from particular racial and ethnic groups—will be disadvantaged by the reliance on government administrative data. In this talk, I will describe some of the work we have done both in the lab and in the community as part of developing, deploying and evaluating a prediction tool currently in use in the Allegheny County Office of Children, Youth and Families.

As machine learning systems become ubiquitous, there is a growing interest in interpretable machine learning -- that is, systems that can provide human-interpretable rationale for their predictions and decisions. In this talk, I'll first give examples of why interpretability is needed in some of our work in machine learning for health, discussing how human input (which would be impossible without interpretability) is crucial for getting past fundamental limits of statistical validation. Next, I'll speak about some of the work we are doing to understand interpretability more broadly: what exactly is interpretability, and how can we assess it? By formalizing these notions, we can hope to identify universals of interpretability and also rigorously compare different kinds of systems for producing algorithmic explanations.

Includes joint work with Been Kim, Andrew Ross, Mike Wu, Michael Hughes, Menaka Narayanan, Sam Gershman, Emily Chen, Jeffrey He, Isaac Lage, Roy Perlis, Tom McCoy, Gabe Hope, Leah Weiner, Erik Sudderth, Sonali Parbhoo, Marzyeh Ghassemi, Pete Szolovits, Mornin Feng, Leo Celi, Nicole Brimmer, Tristan Naumann, Rohit Joshi, Anna Rumshisky, Omer Gottesman, Emma Brunskill, Yao Liu, Sonali Parbhoo, Joe Futoma, and the Berkman Klein Center.

We are all going through a hard time with COVID-19. In fact, COVID-19 is just the tip of the iceberg with many other unsolved biomedical problems, such as cancer early identification and finding side effects of new drugs. These problems seem to be independent of each other and have so far been tackled by different biologists. In this talk, I will argue that, behind these different problems is the same computational challenge, that is, how to understand and predict in never-before-seen situations. In addition to powerful predictive models, what is really needed are tools that generalize well to new drugs, new diseases, and new cohorts.

My talk will focus on our novel machine learning method developed to tackle two kinds of never-before-seen situations: never-before-seen class and never-before-seen cohort. I will first introduce how we classify samples into never-before-seen classes by embedding noisy and large-scale biomedical ontologies, resulting in new discoveries in protein functions, cell types, and rare diseases. Next, I will introduce our solution to understand and characterize a never-before-seen cohort. Instead of finding which features are important, we answer the question of why these features are important using a novel multiscale biomedical knowledge graph. This multiscale knowledge graph is constructed using millions of scientific papers and millions of experimental associations, providing up-to-date and scalable evidence for observations in our multi-scale biomedical world. I will conclude with a vision of future directions for never-before-seen biomedicine.

Recent progress in machine learning (ML) provides us with many potentially effective tools to learn from datasets of ever increasing sizes and make useful predictions. How do we know that these tools can be trusted in critical and high-sensitivity systems? If a learning algorithm predicts the GPA of a prospective college applicant, what guarantees do we have concerning the accuracy of this prediction? How do we know that it is not biased against certain groups of applicants? I will introduce examples of diverse domain applications where these questions are important, as well as statistical ideas to ensure that the learned models apply to individuals in an equitable manner. In recent work with Yaniv Romano, Rina Barber, and Emmanuel Candes, we show how to achieve some fairness objectives we do not need to “open up the black box,” and try understanding its underpinnings. Rather, we discuss broad methodologies — ex. conformal inference — that can be wrapped around any black box to produce results that can be trusted and that are “fair.‘’

Compared with snapshot health information, long-term and high-frequency physiological time series provides health information from the other dimension. I will discuss recently developed signal processing tools in nonlinear-type time-frequency analysis and manifold learning inspired by dealing with this kind of time series. The developed tools will simultaneously handle several challenges when extracting useful biorhythm features—the time series is usually of single channel and composed of multiple oscillatory components with complicated statistical features, like time-varying amplitude, frequency and non-sinusoidal pattern, and the signal quality is often compromised by nonstationary noise and artifact. I will demonstrate how to apply it to some clinical challenges. The established theoretical supports will also be discussed if time permits.

One of the central challenges in genetics is to understand the mapping from genetic variation to phenotypic variation. During the past 15 years, genome-wide association studies (GWAS) have been used to study the genetic basis of a wide variety of complex diseases and other traits. One striking finding from this work has been that for a wide range of complex traits such as height or schizophrenia, even the most important loci in the genome contribute just a small fraction of the phenotypic variance. Instead, most of the variance comes from tiny contributions from tens to hundreds of thousands of variants spread across most of the genome. Our group has argued that these observations do not fit neatly into standard conceptual models of genetics. In recent papers, we have proposed one model to explain this, which we refer to as the "omnigenic" model.

In this talk, I will review our past work in this area, and describe new work that we have done on the genetic basis of three molecular traits---urate, IGF-1, and testosterone---that are biologically simpler than most diseases, and for which we know a great deal in advance about the core genes and pathways. For these molecular traits, we observe huge enrichment of significant signals near genes involved in the relevant biosynthesis, transport, or signaling pathways. However, even these molecular traits are highly polygenic, with most of the variance coming not from core genes, but from thousands to tens of thousands of variants spread across most of the genome. In summary, our models help to illustrate why so many variants affect risk for any given disease

The speaker will present a Bayesian model selection (BMS) approach to detect abnormalities in the data from magnetic resonance imaging guided radiation therapy devices. The BMS method effectively identifies the true abnormalities and suppress the spurious ones. The speaker will discuss several extensions, including detecting structural changes in heat-maps and extracting dynamic resting-state functional connectivity from brain images. The speaker will also introduce a functional censored quantile regression model to describe the time-varying relationship between time-to-event outcomes and corresponding functional covariates. The method was used to analyze the functional relationship between ambulatory blood pressure trajectories and clinical outcome in stroke patients.

Randomized trials in clinical research tend to recruit diverse patients including some individuals who may be unresponsive to the treatment. Here we introduce a methodology to test whether patients near a sweet-spot experience more relative benefit than patients at the extremes. We then demonstrate this methodology based on a randomized trial of defibrillators to reduce all-cause mortality in patients with heart failure. An awareness of this methodology, we suggest, may help for identifying a sweet-spot in a randomized trial where some patients experience more relative benefit than other patients.

This talk explores what it means to develop "grown-up" machine learning algorithms and why they are critical for applications such as medicine and healthcare. I propose three desiderata for grown-up learning algorithms: accountability for mistake/success, transparency in predictions, and flexibility in giving individuals control over their data. These properties turn out to be quite different from what is typically used or studied in machine learning and statistics. I will propose some technical definitions, example applications in healthcare and new algorithms that take an initial step in this direction. We will also teach people (briefly) how to read heart ultrasound.

Clinical laboratory tests are a critical component of the continuum of care and provide a means for rapid diagnosis and monitoring of chronic disease. In this study, we systematically evaluated the genetic basis of 35 blood and urine laboratory tests measured in 358,072 participants in the UK Biobank and identified 1,857 independent loci associated with at least one laboratory test, including 488 large-effect protein truncating, missense, and copy-number variants. We then causally linked the biomarkers to medically relevant phenotypes through genetic correlation and Mendelian Randomization. Finally, we developed polygenic risk scores (PRS) for each biomarker and built multi-PRS models using all 35 PRSs simultaneously. We assessed sex-specific genetic effects and find striking patterns for testosterone with marked improvements in prediction when training a sex-specific model. We found substantially improved prediction of incidence in FinnGen (n=135,500) with the multi-PRS relative to single-disease PRSs for renal failure, myocardial infarction, type 2 diabetes, gout, and alcoholic cirrhosis. Together, our results show the genetic basis of these biomarkers, which tissues contribute to the biomarker function, the causal influences of the biomarkers, and how we can use this to predict disease. For the last 15 minutes of the presentation, I'll briefly touch base on recent progress in COVID-19 host genetics efforts.

A problem that has received attention in recent years is that genetic data is heavily and disproportionately based towards individuals of European origin. Equally apparent upon consideration is that data on disease incidence, progression and treatment outcomes is heavily biased towards that coming from North American and European health care delivery systems. Why would we want to change this situation and bring more diversity into the data that we have available for improving health outcomes and treatment options? If it is a problem that should be solved, what are the barriers to solving it? Finally what are the data analysis challenges that arise as we normalize the contributors to available data.

Estimating causal effects for survival outcomes in the high-dimensional setting is an extremely important topic for many biomedical applications as well as areas of social sciences. We propose a new orthogonal score method and a new Hazards Difference (HDi) estimator, for treatment effect estimation and inference that results in asymptotically valid confidence intervals. We apply our methods to study the treatment effect of radical prostatectomy versus conservative management for prostate cancer patients using the SEER-Medicare Linked Data. Time permitted, I would also like to discuss a new deep learning algorithm we have recently developed for time-to-event data and demonstrate its effectiveness on a number of open real datasets.

Heterogeneity across sub-populations can be beneficially exploited for causal inference and to improve the replicability of feature selections across distributions. The key is to encourage models to exhibit invariance across settings and interventions. The novel methodology potentially offers more robustness and 'causal-oriented' interpretation of results, compared to standard regression and classification methods.

Germline mutations are the main drivers of evolution and the source of many heritable diseases. Understanding the rate and mechanisms by which mutations occur is of paramount importance for studies of human genetics (to interpret the incidence of rare diseases) and evolutionary biology (to date evolutionary events). Despite strong constraints in replication machinery across species, recent studies have documented considerable interspecies and inter-individual variation. Both the overall mutation rate and mutation spectra (the relative proportions of different mutation types) have been shown to differ across species and among human populations. Whole-genome sequencing of pedigrees has enabled direct survey of newly arising mutations (i.e., de novo mutations) and rather than answering questions about the mechanisms involved, it has revealed many puzzles about mutation rate and its evolution. In this talk, I will present analysis of whole genome sequences of pedigrees of humans and other primates and discuss how sex, age and time impact the evolution of mutation rate. These analyses provide insights about the process of mutagenesis and help build a reliable molecular clock for dating evolutionary events.

Metabolism during pregnancy is a dynamic and precisely programmed process, the failure of which can bring devastating consequences to the mother and fetus. To define a high-resolution temporal profile of metabolites during healthy pregnancy and the postpartum period, we carried out an untargeted metabolome investigation of 784 blood samples collected weekly from 30 healthy pregnant Danish women. The study revealed broad metabolome changes during normal pregnancy: of 9,651 detected metabolic features, 4,995 were significantly changed (FDR < 0.05). Of these, 460 annotated compounds (of 687 total) and 34 human metabolic pathways (of 48 total) were significantly altered during pregnancy, revealing a highly choreographed metabolic profile. Using linear models, we built a metabolic clock with metabolites that times gestational age in high accordance with first-trimester ultrasound. Our study represents the first weekly characterization of the human pregnancy metabolome, providing a high-resolution view for understanding human pregnancy with a potential clinical utility.

Logrank/hazard ratio (HR) test/estimation approach has been routinely used in almost all cancer clinical trials with time-to-event outcomes. Although the logrank test is an asymptotically valid nonparametric test, it is not the most powerful test when the pattern of the difference is non-proportional hazards (PH). Also, interpretation of the HR is not obvious when the PH assumption does not hold. For immunotherapy trials, we often see a delayed difference pattern, where the conventional logrank/HR approach is not appropriate for testing equality nor estimating the magnitude of the treatment effect. Restricted mean survival time (RMST)-based analysis has been proposed as an alternative to the logrank/HR approach. It provides a robust and interpretable summary of the treatment effect. However, it is known that the standard RMST-based approach offers lower power than the logrank/HR approach in delayed difference scenarios. We propose a new prespecified RMST-based test and a corresponding treatment effect estimation procedure, particularly when a delayed difference pattern is expected at the design stage. Simulation studies show how effectively the proposed method can detect the treatment difference, compared to the logrank test, various weighted logrank tests, MaxCombo test, standard RMST-based tests, and so on.

Determining the regulatory activity of non-coding genetic variants in specific cell types and cellular contexts is key to understanding the genetic basis of complex disease. In our work we profile the epigenome and annotate the function of disease risk variants using high-throughput molecular assays in combination with statistical approaches including those developed in our lab. I will describe several recent studies where we have used epigenomics to interpret the genetics of complex disease. First, we have mapped regulatory programs of cell types in the human pancreas and numerous other tissues using single cell accessible chromatin, from which we derived disease-relevant cell types and inferred cell type-specific regulatory programs of fine-mapped diabetes and other complex disease risk variants. Second, we annotated regulatory variants at diabetes risk loci by combining high-throughput, allele-specific binding assays of hundreds of TFs with accessible chromatin and dense fine-mapping data. Third, using a novel approach for estimating allelic imbalance we identified disease variants with allelic effects on TF binding and accessible chromatin generated from cells exposed to different in vitro stimuli. Together these studies represent examples of how mapping the epigenome can provide the cell type- and context-specific regulatory activity of non-coding variants and reveal their contribution to disease.

Emerging genomics technology such as CRISPR gene-editing and single-cell sequencing provide us with increasing resolution to unravel the dynamic biology in human health and diseases. Our group aim to develop computation-optimized CRISPR designs for cell barcoding and editing that allows the probing of cancer biology at single-cell resolution. This “experimental + data-analytical” toolkit will enable studies on cancer cell evolution and interaction, as well as potential applications in gene therapy, immunology, and other fields. Ultimately, our long-term goal is to work collaboratively with our collaborating teams to bridge advances in gene-editing, genomics, and data science.

Randomized clinical trials are commonly designed to assess whether a test treatment prolongs survival relative to a control treatment. Increased patient heterogeneity, while desirable for generalizability of results, weakens the ability of common statistical approaches to detect treatment differences, potentially hampering the regulatory approval of safe and efficacious therapies. A novel solution to this problem is proposed. A list of baseline covariates that have the potential to be prognostic for survival under either treatment is pre-specified in the analysis plan (Step 1). At the analysis stage, using observed survival times but blinded to patient-level treatment assignment, the covariate list is shortened via elastic net Cox regression (Step 2) for input into a conditional inference tree algorithm that segments the heterogeneous trial population into subpopulations (strata) of prognostically homogeneous patients (Step 3). After patient-level treatment unblinding, a treatment comparison is done within each formed stratum (Step 4) and stratum-level results are combined for statistical inference using an adaptive strategy (Step 5). The impressive power-boosting performance of our proposed 5-step stratified testing and amalgamation routine (5-STAR) relative to the logrank test and other methods for survival analysis is illustrated using two real datasets and simulations. An R package is available for implementation.

While sample sizes in randomized clinical trials are large enough to estimate the average treatment effect well, they are often insufficient for estimation of treatment-covariate interactions critical to studying data-driven precision medicine. Observational data from real world practice may play an important role in alleviating this problem. One common approach in trials is to predict the outcome of interest with separate regression models in each treatment arm, and recommend interventions based on the contrast of the predicted outcomes. Unfortunately, this simple approach may induce spurious treatment-covariate interaction in observational studies when the regression model is misspecified. Motivated by the need of modeling the number of relapses in multiple sclerosis patients, where the ratio of relapse rates is a natural choice of the treatment effect, we propose to estimate the conditional average treatment effect (CATE) as the relative ratio of the potential outcomes, and derive a doubly robust estimator of this CATE in a semiparametric model of treatment-covariate interactions. We also provide a validation procedure to check the quality of the estimator on an independent sample. We conduct simulations to demonstrate the finite sample performance of the proposed methods, and illustrate the advantage of this approach on real data examining the treatment effect of dimethyl fumarate compared to teriflunomide in multiple sclerosis patients.

Recent years have witnessed increasing empirical successes in reinforcement learning. However, many statistical questions about reinforcement learning are not well understood even in the most basic setting. For example, how many sample transitions are needed and sufficient for estimating a near-optimal policy for Markov decision problem (MDP)? In the first part, we survey recent advances on the methods and complexity for Markov decision problems (MDP) with finitely many state and actions - a most basic model for reinforcement learning. In the second part we study the statistical state compression of general finite-state Markov processes. We propose a spectral state compression method for learning state features and aggregation structures from data. The state compression method is able to “ sketch” a black-box Markov process from its empirical data, for which both minimax statistical guarantees and scalable computational tools are provided. In the third part, we propose a bilinear primal-dual pi learning method for learning the optimal policy of MDP, which utilizes given state features. The method is motivated from a saddle point formulation of the Bellman equation. Its sample complexity depends only on the number of parameters and is variant with respect to the dimension of the problem, making high-dimensional reinforcement learning possible using “small” data. 

Randomized controlled trials (RCT's) are an indispensable source of information about efficacy of treatments in almost any disease area. With the availability of multiple treatment options, comparative effectiveness research (CER) is gaining importance for better and informed health care decisions. However design and analysis of effectiveness trial is much more complex than the efficacy trial. The effect of including an active comparator arm/s in a RCT is immense. This gives rise to superiority and non-inferiority trials. The non-inferiority (NI) RCT design plays a fundamental role in CER, which will be also focus of this talk. In the past decade many statistical methods have been developed, though largely in the Frequentist setup. However, availability of historical placebo-controlled trial is useful and if integrated in the current NI trial design, can provide better precision for CER. This may reduce sample size burden and improves statistical power significantly in current trial. Bayesian paradigm provides a natural path to integrate historical as well as current trial data via sequential learning in the NI setup. In this talk we will discuss both fraction margin and fixed margin based Bayesian approach for three-arm NI trial. We will also discuss some interesting open problems related to CER using NI trial.

Tissue composition is a major determinant of phenotypic variation and a key factor influencing disease outcomes. Although single-cell RNA sequencing has emerged as a powerful technique for characterizing cellular heterogeneity, it is currently impractical for large sample cohorts and cannot be applied to fixed specimens collected as part of routine clinical care. Over the last decade, a number of computational techniques have been described for dissecting cellular content directly from genomic profiles of mixture samples. In this talk, I will review key computational and statistical considerations for “digital cytometry” applications. I will also discuss basic and translational efforts from our group to leverage cell signatures derived from diverse sources, including single-cell reference profiles, to infer cell type abundance and cell type-specific gene expression profiles from bulk tissue transcriptomes. Digital cytometry has the potential to augment single cell profiling efforts, enabling cost-effective, high throughput tissue characterization without the need for antibodies, disaggregation, or viable cells.

Across modern biology, open-source scientific software is increasingly critical for progress. At the Chan Zuckerberg Initiative, we are supporting this ecosystem through both grantmaking and software development. I'll describe specific ongoing efforts for the storage, analysis, and visualization of cell biology sequencing and imaging data. I'll also highlight our broader ideas for building and supporting data sharing and open ecosystems for computational biology more generally.

The HIV Prevention Trial Network (HPTN) 052 Study is a Phase III, controlled, randomized clinical trial to assess the effectiveness of immediate versus delayed antiretroviral therapy strategies on sexual transmission of HIV-1 (Cohen, et al., 2016). It was selected by the “Science Magazine” as the Scientific Breakthrough of the Year 2011 (Alberts, 2011). In this talk, we will focus on the design and methods that underlie this landmark study in HIV Treatment-as-Prevention, and discuss the lessons that we have learned for future prevention research. References: Alberts, B (2011) Science breakthroughs, Science, 334: 1604; Cohen, MS, Chen, YQ, McCauley, M, et al. (2016) Antiretroviral therapy for the prevention of HIV transmission. New England Journal of Medicine, 375: 830-839.

Recently, there has been a surge in applying statistical learning methods in healthcare, to build models for predicting adverse outcomes from patient covariates. These predictions are then used to optimize allocation of scarce resources or treatment decisions. However, when the treatments are new, decision-making should optimize a trade-off between two objectives: (1) learning decision outcomes as functions of individual-specific covariates (exploration) and (2) maximizing benefit of the decisions. Current literature on this problem, theory of contextual multi-armed bandits, focuses on algorithms that rely on forced-exploration to address this trade-off. However, forced-exploration can be considered costly or unethical in certain decision-making tasks (e.g., hospital quality improvement initiatives). In this talk, we first introduce an algorithm that leverages freeexploration from patient covariates and achieves rate optimal objective. We also show, empirically, that the algorithm significantly reduces exploration, compared to the existing benchmarks. Next, we focus on settings when past data on decision outcomes is available. Motivated by recent literature on low-rank matrix estimation, we design algorithms that avoid unnecessary exploration by targeting the learning towards shared similarities among decisions or patients. We then demonstrate performance of the proposed methods to estimate the personalized effect of a glucose inhibitor drug (Metformin) for pre-diabetic treatment.

Rapid advances in genomic technologies have led to a wealth of diverse data, from which novel discoveries can be gleaned through the application of robust statistical and computational methods. Here we describe GeneFishing, a computational approach to reconstruct context-specific portraits of biological processes by leveraging genegene co-expression information. GeneFishing incorporates multiple high-dimensional statistical ideas, including dimensionality reduction, clustering, subsampling and results aggregation, to produce robust results. To illustrate the power of our method, we applied it using 21 genes involved in cholesterol metabolism as “bait”, to “fish out” (or identify) genes not previously identified as being connected to cholesterol metabolism. Using simulation and real datasets, we found the results obtained through GeneFishing were more interesting for our study than those provided by related geneprioritization methods. In particular, application of GeneFishing to the GTEx liver RNAseq data not only re-identified many known cholesterol-related genes, but also pointed to glyoxalase I (GLO1) as a novel gene implicated in cholesterol metabolism. In a follow-up experiment, we found that GLO1 knock-down in human hepatoma cell lines increased levels of cellular cholesterol ester, validating a role for GLO1 in cholesterol metabolism. In addition, we performed pan-tissue analysis by applying GeneFishing on various tissues and identified many potential tissue-specific cholesterol metabolism related genes. GeneFishing appears to be a powerful tool for identifying novel related components of complex biological systems and may be employed across a wide range of applications.

With the development of rapid, low-cost and readily available sequencing technologies, there is a need for quantitative methods to help interpret sequence datasets and relate them to the dynamics of biological systems. Trees (in the sense of graphs with no cycles) are a mainstay of how we represent and understand sequence data. I will introduce several flavours of trees with their motivating applications, and will describe a metric -- in the sense of a true distance function -- on unlabelled binary trees; this metric is derived from polynomials on the unlabelled trees. In the second part of the talk I will describe inference tools using trees, in the context of infectious disease: we use a mapping between phylogenetic trees and transmission trees to construct a Bayesian MCMC approach to estimate who infected whom and when. I will describe extensions of this inference approach to simultaneous reconstruction of outbreaks in different clusters, and conclude with a description of open problems and challenges in this area.

Brain functional connectivity maps the intrinsic functional architecture of the brain through correlations in neurophysiological measures of brain activities. Accumulated evidences have suggested that it holds crucial insights of pathologies of a wide range of neurological disorders. Brain functional connectivity analysis is at the foreground of neuroscience research, and is drawing increasing attention in the statistics field as well. A connectivity network is characterized by a graph, where nodes represent brain regions, and links represent statistical dependence that is often encoded by partial correlation. Such a graph is inferred from the matrixvalued neuroimaging data such as electroencephalography and functional magnetic resonance imaging. In this talk, we examine a number of statistical problems arising in brain connectivity analysis, including multigraph penalized estimation, graph-based hypothesis testing, and dynamic connectivity network modeling.