Two new papers on predictability of symptom networks

This guest post was written by Jonas Haslbeck (jonas.haslbeck@gmail.com) who is a PhD student in the Psychosystems lab at the University of Amsterdam.

Symptom networks have become popular in clinical psychology because they promise to provide insights into the topology and dynamics of mental disorders. Most analyses so far focused on the structure of the symptom network – how are symptoms connected, and which symptoms are most central in the network?

Here we present two papers that look into an additional characteristic of network models – the predictability of each node by all other nodes in the network. Predictability is interesting for two reasons:

1. Predictability allows to judge the relevance of edges connected to a symptom; if symptom A is determined to 90% by its neighbors, its edges will be practically more relevant than if only 3% is determined by its neighbors. This is especially interesting in clinical practice, where one has to judge whether an intervention on a symptom via the symptom network is feasible.
2. The predictability of all nodes taken together gives an indication of how self-determined (a part of) the network is: if the overall predictability is high, the symptom network is largely self-determined, and if the overall predictability is low, the symptom network is largely determined by other variables that are not included in the analysis.

In the first paper¹, we show how to compute predictability using the R-package mgm and how to use the qgraph package to incorporate predictability measures in a typical network visualization. The following figure visualizes a mixed graphical model estimated on the data from Fried et al. (2015)². The blue ring shows the proportion of explained variance (for continuous nodes). For the binary variable ”loss”, the orange part of the ring indicates the accuracy of the intercept model. The red part of the ring is the additional accuracy achieved by neighboring variables.

In the second paper³, we discuss the utility of predictability in psychopathological research and provide a first peak into the predictability of symptom networks by reanalyzing 25 datasets from 18 published papers in the field. The figure below summarizes the distribution of predictability measures for datasets with continuous (left) and binary (right) data. The x-axis shows the predictability measure (ranging from 0 to 1): ‘normalized accuracy’ for binary variables and ‘proportion of explained variance’ for continuous variables. The box plot whiskers show 1.5 times the Interquantile Range (IQR).

It is worth pointing out that predictability may be highly interesting for many other psychological network disciplines beyond psychopathology research such as personality or attitude research.

Abstract paper 1

Haslbeck, J., & Waldorp, L. (2017). How well do network models predict future observations? On the importance of predictability in network models. Behavior Research Methods, 1–17 (PDF).

Network models are an increasingly popular way to abstract complex psychological phenomena. While studying the structure of network models has led to many important insights, little attention has been paid to how well they predict observations. This is despite the fact that predictability is crucial for judging the practical relevance of edges: for instance in clinical practice, predictability of a symptom indicates whether an intervention on that symptom through the symptom network is promising. We close this methodological gap by introducing nodewise predictability, which quantifies how well a given node can be predicted by all other nodes it is connected to in the network. In addition, we provide fully reproducible code examples of how to compute and visualize nodewise predictability both for cross-sectional and time series data.

Abstract paper 2

Haslbeck, J. M. B., & Fried, E. I. (2017). How predictable are symptoms in psychopathological networks? A reanalysis of 18 published datasets. Psychological Medicine, 1-10 (PDF).

Network analyses on psychopathological data focus on the network structure and its derivatives such as node centrality. One conclusion one can draw from centrality measures is that the node with the highest centrality is likely to be the node that is determined most by its neighboring nodes. However, centrality is a relative measure: knowing that a node is highly central gives no information about the extent to which it is determined by its neighbors. Here we provide an absolute measure of determination (or controllability) of a node – its predictability. We introduce predictability, estimate the predictability of all nodes in 18 prior empirical network papers on psychopathology, and statistically relate it to centrality.
We carried out a literature review and collected 25 datasets from 18 published papers in the field (several mood and anxiety disorders, substance abuse, psychosis, autism, and transdiagnostic data). We fit state-of-the-art network models to all datasets, and computed the predictability of all nodes.
Predictability was unrelated to sample size, moderately high in most symptom networks, and differed considerable both within and between datasets. Predictability was higher in community than clinical samples, highest for mood and anxiety disorders, and lowest for psychosis.
Predictability is an important additional characterization of symptom networks because it gives an absolute measure of the controllability of each node. It allows conclusions about how self-determined a symptom network is, and may help to inform intervention strategies. Limitations of predictability along with future directions are discussed.

Footnotes

1. Haslbeck, J., & Waldorp, L. (2017). How well do network models predict future observations? On the importance of predictability in network models. Behavior Research Methods, 1–17, advance online print (PDF)
2. Fried, E. I., Bockting, C., Arjadi, R., Borsboom, D., Amshoff, M., Cramer, A. O., Epskamp, S., Tuerlinckx, F., Carr, D., & Stroebe, M. (2015). From loss to loneliness: The relationship between bereavement and depressive symptoms. Journal of Abnormal Psychology, 124(2), 256 (PDF).
3. Haslbeck, J. M. B., & Fried, E. I. (2017). How predictable are symptoms in psychopathological networks? A reanalysis of 18 published datasets. Psychological Medicine, 1-10, advance online print (PDF).