The post Network replicability: a cross-cultural PTSD study across of clinical datasets appeared first on Psych Networks.

]]>This is how the four networks look like:

I will discuss below (1) why research on replicability and generalizability is important, (2) why we should start focusing on estimating network and factor models in clinical datasets instead of community samples, and (3) discuss the paper in more detail.

We’ve worked hard in the last year to tackle the challenge of stability or accuracy of network models. As explained in a recent tutorial blog post, the R-package *bootnet* [1] can be used to investigate how stable state-of-the-art network models such as the Ising Model or the Gaussian Graphical Model are. The package was updated recently and now can also be used for relative importance networks, and we are working hard to be able to incorporate time-series models in the future.

Now that we can look into the stability of network models — and can show that network parameters can be estimated accurately in large samples — the next question is whether data-driven and exploratory network models generalize and replicate across different samples. Interestingly, there is no single published paper that I am aware of that has done so specifically. Note that a few papers exist that compared subsamples of the same dataset, such as groups of individuals with different substance abuse disorders [2].*

So about a year ago, I set out to write a paper on network replicability, and found a dozen of collaborators in Denmark, the Netherlands, and Italy who were willing to help with this endeavour.

In addition to replicability, there is a second main limitation of the network literature so far: similar to the SEM literature, the majority of prior studies were carried out in community or subclinical samples, rendering network structures in clinical samples largely unknown. This is most pronounced for the PTSD literature where more than 10 network papers have been published, but no single paper in a purely clinical dataset. Which is why we investigated the replicability of PTSD networks in 4 clinical samples.

I uploaded the preprint of the paper yesterday. The repository not only includes the paper itself, but also the covariance matrix of the four datasets, along with all network parameter estimates, symptoms means and standard deviations, and many other R objects to make the paper fully reproducible. While we cannot share the original datasets, the covariance matrices are sufficient to estimate network and factor models, and we encourage re-analysis of the data.

What did we do, and what did we find? We estimated state-of-the-art Gaussian Graphical Models in four clinical datasets of traumatized patients seeking treatment. For the first time, to our knowledge, we estimated four networks *jointly*, using the Fused Graphical Lasso (FGL) [3] implemented by Giulio Costantini who collaborated on the paper. The FGL improves network estimates by exploiting similarities among different groups in case such similarities emerge; otherwise, networks are estimated independently. We then compared the resulting networks in terms of network structure and centrality indices. Here is a short summary:

The network approach to psychopathology understands mental illness like Posttraumatic Stress Disorder (PTSD) as networks of causally interacting symptoms. The prior literature is limited in three aspects: studies estimated networks in one sample each, leaving open the question whether networks replicate across samples; studies estimated networks in primarily small samples that may lack power for reliable estimation; and studies examined community or subclinical samples, rendering the network structure in clinical samples unknown. In this cross-cultural multisite study, we estimated state-of-the-art regularized partial correlation networks of 16 PTSD symptoms across four datasets of traumatized patients (total N=2,782). Considerable similarities emerged, with high correlations between network structures (0.62 to 0.74) and centrality estimates (0.63 to 0.75). Only 1.7% to 6.7% of the 120 edges differed across networks. Despite sample differences, networks showed substantial similarities, suggesting that PTSD symptoms may be associated in similar ways. We discuss implications for generalizability and replicability.

First, it was interesting to see that symptom profiles were fairly similar across datasets, despite the very different composition and background of the four samples (ranging from a sample of predominantly male military veterans to another dataset of severely traumatized refugees with up to 30% psychotic symptoms). The plot below also contains standard deviations because there were some ceiling effects in dataset four, with very severe symptomatology and lower standard deviations, with a negative correlation of mean and standard deviation which can be nicely seen in the plot (details in the paper).

Second, network structures were not exactly the same: the omnibus network comparison test (with the null-hypothesis that all 120 edges per comparison are exactly identical) was significant for all pairs of networks. Overall, however, considerable similarities emerged, and correlations between network structures and centrality indices were high (see paper for more details).

Third, we estimated (a) a cross-sample network and (b) a cross-sample variability network. These depict similarities and differences across the four datasets.

Overall, we were surprised by the similarities of both symptom profiles and network structures across the four datasets, given the considerable differences in sample compositions. We conclude the paper with a call for more replicability work in psychopathology factor and network models:

We therefore conclude that investing time in more thoroughly conducted cross-sample studies for both network and factor models is warranted in order to facilitate insights about replicability and generalizability. We hope the present paper will encourage more researchers to do so.

» Fried, E. I., Eidhof, M. B., Palic, S., Costantini, G., Huisman-van Dijk, H. M., Bockting, C. L. H., Engelhard, I., Armour, C., Nielsen, A. B. S., Karstoft, K. (submitted). Replicability and generalizability of PTSD networks: A cross-cultural multisite study of PTSD symptoms in four trauma patient samples. DOI: 10.17605/OSF.IO/2T7QP. (Preprint and Supplementary Materials)

Introduction. The network approach to psychopathology understands disorders like Posttraumatic Stress Disorder (PTSD) as networks of mutually interacting symptoms. The prior literature is limited in three aspects. First, studies have estimated networks in one sample only, leaving open the crucial question of replicability and generalizability across populations. Second, many prior studies estimated networks in small samples that may not be sufficiently powered for reliable estimation. Third, prior PTSD network papers examined community or subclinical samples, rendering the PTSD network structure in clinical samples unknown. In this cross-cultural multisite study, we estimate and compare networks of PTSD symptoms in four heterogeneous populations of trauma patients with different trauma-types, including civilian-, refugee-, combat-, post-war off-spring-, and professional duty-related trauma.

Methods. We jointly estimated state-of-the-art regularized partial correlation networks across four datasets (total N=2,782), and compared the resulting networks on various metrics such as network structure, centrality, and predictability.

Results. Networks were not exactly identical, but considerable similarities among the four networks emerged, with moderate to high correlations between network structures (0.62 to 0.74) and centrality estimates (0.63 to 0.75); only few edges differed significantly across networks.

Conclusion. Despite differences in culture, trauma-type and severity of the four samples, the networks showed substantial similarities, suggesting that PTSD symptoms may be associated in similar ways. We discuss implications for generalizability and replicability. A step-by-step tutorial is available in the supplementary materials, including all analytic syntax and all relevant data to make the paper fully reproducible.

[2] Rhemtulla, M., Fried, E. I., Aggen, S. H., Tuerlinckx, F., Kendler, K. S., & Borsboom, D. (2016). Network analysis of substance abuse and dependence symptoms. Drug and Alcohol Dependence, 161, 230–237. http://doi.org/10.1016/j.drugalcdep.2016.02.005

[3] Guo, J., Levina, E., Michailidis, G., & Zhu, J. (2011). Joint estimation of multiple graphical models. Biometrika, 98(1), 1–15. http://doi.org/10.1093/biomet/asq060

* There is also a recent preprint by Forbes et al. to be published in the *Journal of Abnormal Psychology* that aimed to tackle replicability, but it makes more sense to discuss the paper once the commentaries on the paper are published that — to my knowledge — will point to substantial problems in the analyses.

The post Network replicability: a cross-cultural PTSD study across of clinical datasets appeared first on Psych Networks.

]]>The post Public time-series data of 40 outpatients (21 items, 130 measurements) appeared first on Psych Networks.

]]>As readers of this blog are likely aware, network analysis represents a new and exciting paradigm that holds the potential to better delineate the structure and dynamic organization of psychopathology. However, as I’ve written about elsewhere, the symptomatology of common psychiatric disorders is composed of time-varying phenomena, occurring within individuals. This presents sizable challenges to research generally, and network analysis specifically. Quite simply, time-varying phenomena can only be effectively observed through time-varying data collection and within-individual (i.e. idiographic) processes can only be uncovered via idiographic research methodologies. Peter Molenaar has written on these issues extensively and I encourage everyone to read his manifesto on idiographic science. To summarize his position, inter-individual variation and intra-individual variation are inherently unrelated, an assertion that Molenaar supports through mathematical proof. As a consequence, he argues, the burden of proof should fall on researchers to demonstrate that the nomothetic and idiographic are at least consonant – if not identical – if we want to generalize the former to the latter. Phenomena for which inter- and intra-individual processes (e.g. variance-covariance) are equivalent are known as ergodic. If a given statistical relationship exhibits the same relative strength and direction across both idiographic and nomothetic paradigms, then conclusions from one can be extrapolated to the other. However, Molenaar has argued that most psychological processes are unlikely to be ergodic. In fact, James Boswell and I found evidence that the well-known (and widely-replicated) positive correlation between depression and anxiety is not reliably reproduced within individuals, and may even exhibit a negative relationship over time (see Fisher & Boswell, 2016).

Thus, at the risk of denigrating or minimizing the groundbreaking, paradigm-shifting, and altogether important work that has been published in the past several years on cross-sectional and/or nomothetic data, I would argue that researchers should endeavor to collect intensive repeated measures data in order to conduct idiographic time series analyses of pathologic phenomena. While our cross-sectional and nomothetic work may help to establish new methods, develop rationales, stimulate hypothesis generation, and foment interest, we must be careful about the degree to which we use nomothetic research to understand the phenomenology and behavior of individuals. Ambulatory and ecological momentary assessments can be readily employed to collect hundreds of observations of thoughts, feelings, and actions, which can then be leveraged to produce multivariate time series for dynamic, intraindividual analysis.

Over the past three years, my lab has collected such data. In the course of conducting an open trial for a personalized, modular cognitive-behavioral therapy (see Fisher & Boswell, 2016 for more details), we have interviewed and enrolled 40 individuals with primary, DSM-based generalized anxiety disorder (GAD) and/or major depressive disorder (MDD). Prior to therapy, participants complete an intensive repeated measures paradigm in which they report their momentary experience of 21 items related to mood and anxiety psychopathology (such as avoidance behavior, positive and negative affect, anhedonia, depressed mood, worry, etc.). Surveys are completed 4x/day for approximately 30 days. To date, the average number of completed surveys has been 130.43 (SD = 19.27), with a range from 87 to 212.

Recently, we have developed a method for leveraging both structural equation modeling (SEM) and network analytic methodologies in order to analyze contemporaneous and time-lagged relationships in intraindividual mood and anxiety syndromes. This work is currently under review, however the revised manuscript and supporting documents can be found on the Open Science Framework at https://osf.io/zefbc/. While responding to reviewer feedback, it became apparent that the best way to present our methods and results – to maximize transparency, clarity, and technical explication – was to share the raw data alongside the code for data preparation and analysis. Thus, at the OSF location noted above, readers can find the complete multivariate time series for the 40 participants reported in the manuscript under review. In addition, readers will find the revised manuscript and step-by-step instructions for carrying out the analyses described in the manuscript. Participant data such as age, sex, ethnicity, and diagnosis can be found in Table 1.

To our knowledge this is the largest and most detailed data set of its kind and I am sincerely excited to share it with interested parties. As an advocate for idiographic science, I believe that seeing is believing. Therefore, it is my hope that working with these data will help to illuminate the necessity and potential impact of idiographic work. Our lab welcomes any and all potential collaborators. Feel free to contact me at *afisher@berkeley.edu* with any queries or comments. For those interested in working with these data, please note that there are currently two manuscripts in preparation: One on the idiographic factor structure (p-technique) of each time series and another examining the predictability of nodes in both contemporaneous and time-lagged networks. As other projects develop, we will endeavor to update this post.

Enjoy!

The post Public time-series data of 40 outpatients (21 items, 130 measurements) appeared first on Psych Networks.

]]>The post Public dataset with 1478 timepoints over 239 consecutive days appeared first on Psych Networks.

]]>The paper was published in the new Journal of Open Psychology Data that you may want to keep an eye on. They publish datasets, which incentivizes sharing of data because you can get cited for it. #openscience

The paper encompasses a dataset and a detailed description thereof. The data were used in a prior paper by Marieke Wichers and colleagues, who identified that the dynamic network of one remitted depression person showed signs of *critical slowing down* before he transitioned from a healthy to a depressed state. The authors argue that this could be an early warning signal which may enable us to predict such phase transitions in the future.

So how do the data look like? It is a n=1 case-study time-series dataset with 50 variables and 1478 timepoints assessed over 239 consecutive days. Many items were collected 10 times per days, others daily (sleep quality and mood) or weekly (depression checklist). Here a visualization of the data (AD = antidepressant) from Wichers et al. 2016.:

The reuse potential of this dataset is considerable. Not only does it exceed any other studies I am aware of in terms of measurement points, but it also features a critical transition where the patient relapses into depression. As Kossakowski and colleagues state: In general, the data “are suitable for various time-series analyses and studies in complex dynamical systems […].” Specifically, the data can be used for three main purposes. “First, the data are extremely suitable for researchers to validate new methods for predicting the onset of a critical transition. Second, there have been recent developments into estimating time-varying networks. This data can be used as an empirical example to show how time-varying networks can be estimated and how the network develops over time. Lastly, since items were measured at different time scales, this dataset can aid research that aims to combine (time-series) data from different time scales.”

Abstract

We present a dataset of a single (N = 1) participant diagnosed with major depressive disorder, who completed 1478 measurements over the course of 239 consecutive days in 2012 and 2013. The experiment included a double-blind phase in which the dosage of anti-depressant medication was gradually reduced. The entire study looked at momentary affective states in daily life before, during, and after the double-blind phase. The items, which were asked ten times a day, cover topics like mood, physical condition and social contacts. Also, depressive symptoms were measured on a weekly basis using the Symptom Checklist Revised (SCL-90-R). The data are suitable for various time-series analyses and studies in complex dynamical systems.

Kossakowski, J.J. et al., (2017). Data from ‘Critical Slowing Down as a Personalized Early Warning Signal for Depression’. Journal of Open Psychology Data. 5(1). DOI: http://doi.org/10.5334/jopd.29

The post Public dataset with 1478 timepoints over 239 consecutive days appeared first on Psych Networks.

]]>The post Tutorial: how to review psychopathology network papers appeared first on Psych Networks.

]]>- The nature of the investigated sample
- Item variability
- Selection effects and heterogeneity
- Appropriate correlations among items
- Publish your syntax and some important output
- Item content
- Stability
- Considerations for time-series models
- Further concerns and new developments

My main focus here is on network papers based on between-subjects cross-sectional data, although some of the points below also apply to within-subjects time-series network models. Note that these are all personal views, and I am sure other colleagues focus on other aspects when they review papers. Feedback or comments are very welcome.

In the structural equation modeling (SEM) literature, researchers often investigate the factor structure of instruments that screen for psychiatric disorders. Frequently, data come from general population samples which predominantly consist of healthy individuals. This means that a conclusion such as “we identified 5 PTSD factors” seems difficult to draw, seeing that only few people in the sample actually met the criteria for a PTDS diagnosis.

The same holds for network models: I have reviewed many papers recently that draw inferences about the network structure of depression, or about comorbidity, but they investigate largely healthy samples. I don’t think that’s a good idea for somewhat obvious substantive reasons: don’t draw conclusions about a disorder if you don’t study patients with that disorder. But there are also two statistical reasons why the above may be a bad idea, to which I will get below.

One argument against my position is that when we study intelligence or neuroticism in SEM, we investigate the factor structure of these constructs in broad samples, and don’t focus on populations of very intelligent or very neurotic people. That is true, and in that sense what you should do here depends a bit on how you understand a given mental disorder. If you think depression is a continuum between healthy and sick, with a somewhat arbitrary threshold, maybe you’re ok studying a general population samples to draw conclusions about depression. If you think depressed people are those with a DSM-5 diagnosis of depression, it may make less sense if you study depression when only 5% of your sample meet these criteria. My main point is that as a reviewer, I’d like to see that you think about this problem.

Community samples may have very low levels of psychopathology, which can make the study of psychopathology in such samples difficult. If the item means are too low, the variability of the items becomes very small, and that can lead to estimation problems (an item without variability will not be connected in your estimated network because it cannot covary with other items). Note that technically this is not really an estimation problem – you can estimate your network just fine – but you may want to be aware of this when interpreting your network. I wrote a brief blogpost about this topic recently inspired by a paper from Terluin et al. where you can find more information.

The same problem can occur in samples with very severe levels of psychopathology or in case you have selection effects. For instance, we recently submitted a paper where the sample consisted of patients with very severe recurrent Major Depression. The mean of the “Sad Mood” item was 0.996 (item range 0-1), so we decided to drop it from the network analysis. Another example is the PTSD A criterion that states: “trauma survivors must have been exposed to actual or threatened death, serious injury, or sexual violence”. It makes no sense to include this item in a network analysis of PTSD patients because every person endorses this item.

Independent of what populations you study, differential item variability can pose a challenge to the interpretation of networks (items with low variability will never end up being central items); that is, if an item in your network has a low variability and low centrality, it’s not clear if that item also would have a low centrality in a population in which it has a variability comparable to the other items. So I recommend you always check and report both means and variances of all your items. This can also be a problem when you, for instance, compare healthy and sick groups with each other. Multiple papers have now found that the connectivity or density of networks in depressed populations is higer than in healthy samples; connectivity is the sum of all absolute edge weights in a network. But this may be driven by differential variability: in healthy samples, the standard deviations of items will be smaller, which means items cannot be as connected as they are in the depressed sample.

A second problem with studying mental disorders in healthy people is that there are good reasons to assume that the factor or network structure of psychopathology symptoms differs between healthy and sick people. We have found very convincing and consistent evidence for this phenomenon in Major Depression, across four different rating scales and two large datasets. And there are also good substantive reasons why this could be the case (e.g. 1, 2, 3).

In healthy samples, you may thus often end up drawing conclusions – for instance about the dimensionality or network structure of depression symptoms – that would not replicate in a sample of depressed patients.

Related to this is the problem of selection or conditioning on sum-scores. This is a tricky problem, and I’ve been trying to wrap my head around this for over a year now. Essentially, if you select a subpopulation (e.g. people with depression) on a sum-score of items that you then put into a network, you get a biased network structure. This is shown analytically in a great paper by Bengt Muthén 1989 to which Dylan Molenaar pointed me a few months ago.

It’s not always easy to decide what type of correlation coefficient is appropriate for what type of data. In psychopathology research, data are often ordered-categorical and skewed. There are different ways to deal with this type of data, and Sacha Epskamp recently asked the following tricky question to students in the Network analysis course at University of Amsterdam:

*“Often, when networks are formed on symptom data, the data is ordinal and highly skewed. For example, an item ‘do you frequently have suicidal thoughts’ might be rated on a three point scale: 0 (not at all), 1 (sometimes) and 2 (often). Especially in general population samples, we often see that the majority of people respond with 0 and only few people respond with a 2. This presents a problem for network estimation, as such data is obviously not normally distributed. Which of the following methods would you prefer to analyze such highly skewed ordinal data?” *

We see the following 3 possibilities:

- You can dichotomize your data, but you will lose information.
- You can use polychoric or Spearman correlations that usually deal well with skewed ordinal variables.
- You can transform your data, for instance using the nonparanormal transformation. But that only works in certain cases.

Personally, I tend to dichotomize in case I only have 3 categories and items are skewed (e.g. this paper). With 4 or more categories, the polychoric correlation seems to work best, and there is also some unpublished simulation work showing this. What I want to see when I review papers is that researchers are aware of this problem, thought about it and tried to find out whether what they are doing with their data is ok. For instance, I usually do both polychoric and Spearman, and if there are dramatic differences, I have a problem I need to solve.

What you can always deposit online, or publish as supplementary materials, is your R code. This will make your results reproducible. What you also should publish are the means and standard deviations of your items. In the best case, you can simply publish your data, but that is often not possible. If you have ordinal or metric data, you can make your networks reproducible by publishing the covariance matrix of your items (because we use this matrix as input when we estimate the Gaussian Graphical Model). In case of the Ising Model, we need the raw data to estimate the network model, so without data your network will not be reproducible. However, you can still publish your model output (i.e. the edges and threshold parameters) so that your graph itself becomes reproducible, and you can also publish the tetrachoric correlations among items for some more insights.

I have only recently come to pay attention to this more, but it seems important to consider the content of the instrument you want to investigate as a network of items. That sounds both obvious and vague, but in some of my prior work, I simply threw all items of a rating scale into a network analysis. For instance, consider this network from a paper we published on bereavement:

Thinking about this now, it seems that sad mood and feeling depressed are fairly similar to each other: do we really want to include both in one network? Or would it be better to represent these items as one node, each by averaging them, or estimating a latent variable? Angélique Cramer and me wrote about this in more detail in a paper entitled “Moving forward: challenges and directions for psychopathological network theory and methodology” that is currently under revision:

“*We see two remaining challenges pertaining to the topic of constituent elements: 1) what if important variables are missing from a system, and 2) what to do with nodes that are highly correlated and may measure the same construct (such as ‘sad mood’ and ‘feeling blue’)?*“

You can find the relevant section on pp. 17-19. I don’t have a solution, but it seems an important topic to pay attention to, especially since we work with partial correlation networks, and I wonder what remains of the association between sad mood and e.g. insomnia after we partial out depressed mood and feeling blue.

We have written about this a lot in the past, so I will only briefly reiterate: please check the stability, accuracy, and robustness of your network models (blog post; paper). It really helps the editor and reviewers of the paper to gauge its relevance and implications, but also helps readers to get a better grasp of the results. Conclusions should be proportional to evidence that is presented, and robust models certainly help with stronger evidence. I also gave a short presentation about robustness at APS 2016, and you can find a whole collection of updated slides on network robustness in the online materials of the network analysis workshop we have just 2 weeks ago.

I have only reviewed a few time-series papers, and other people are much better suited go give feedback here. A good start for state-of-the-art models, model assumptions, and pitfalls are recent papers by Kirsten Bulteel, Jonas Haslbeck, Laura Bringmann, and Noémi Schuurman.

You can find a number of additional concerns and topics I wonder about in our draft on challenges to the network approach, but these currently play do not a major role when I review papers. And maybe you’d like to add important concerns to the comments below. In general, please be careful with drawing causal inference from cross-sectional data, and keep in mind that between-subjects and within-subjects effects may be different from each other.

If you want to try out something novel that will hopefully become state-of-the-art in the next half year, check out the predictability metric that Jonas Haslbeck developed. This will result in something akin to R^2 of each node explain by all other nodes (indicated by the grey area around the nodes):

And it never hurts to have a research question of course: something you’re interested in, a hypothesis. I find network papers stronger and more interesting if it’s more than just applying

`estimateNetwork(data, default="EBICglasso")`

to a new dataset or disorder. The same applies to SEM papers as well of course.

Some of these concerns come from discussions with colleagues such as Sacha Epskamp, Angélique Cramer, Denny Borsboom, Jonas Haslbeck, Claudia van Borkulo, Kamran Afzali, and Aidan Wright. So kudos to them and all other colleagues who commented on this blog post, and who have helped me grasp these issues in the last 2 years.

The post Tutorial: how to review psychopathology network papers appeared first on Psych Networks.

]]>The post The meaning of model equivalence: Network models, latent variables, and the theoretical space in between appeared first on Psych Networks.

]]>Recently, an important set of equivalent representations of the Ising model was published by Joost Kruis and Gunter Maris in Scientific Reports. The paper constructs elegant representations of the Ising model probability distribution in terms of a network model (which consists of direct relations between observables), a latent variable model (which consists of relations between a latent variable and observables, in which the latent variable acts as a common cause), and a common effect model (which also consists of relations between a latent variable and observables, but here the latent variable acts as a common effect). The latter equivalence is a novel contribution to the literature and a quite surprising finding, because it means that a formative model can be statistically equivalent to a reflective model, which one may not immediately expect (do note that this equivalence need not maintain dimensionality, so a model with a single common effect may translate in a higher-dimensional latent variable model).

However, the equivalence between the ordinary (reflective) latent variable models and network models has been with us for a long time, and I therefore was rather surprised at some people’s reaction to the paper and the blog post that accompanies it. Namely, it appears that some think that (a) the fact that network structures can mimic reflective latent variables and vice versa is a recent discovery, that (b) somehow spells trouble for the network approach itself (because, well, what’s the difference?). The first of these claims is sufficiently wrong to go through the trouble of refuting it, if only to set straight the historical record; the second is sufficiently interesting to investigate it a little more deeply. Hence the following notes.

The equivalence between statistical network models (more specifically, random fields like the Ising model) and latent variable models (e.g., the IRT model) is not actually new. Peter Molenaar in fact suspected the equivalence as early as 2003, when he was still in Amsterdam, stating that “it always struck me that there appears to be a close connection between the basic expressions underlying item-response theory and the solutions of elementary lattice fields in statistical physics. For instance, there is almost a one-to-one formal correspondence of the solution of the Ising model (a lattice with nearest neighbor interaction between binary-valued sites; e.g., Kindermann et al. 1980, Chapter 1) and the Rasch model.” (see p. 82 of this book). Peter never provided a formal proof of his assertion, as far as I know, but clearly the idea that network models and other dynamical models bear a close relation to latent variable models was already in the air back then. I remember various lively discussions on the topic.

The connection between latent variables and networks got kick-started a few years later, when Han van der Maas walked into my office. At the time, he was thinking a lot about the relation between general intelligence and IQ subtest scores, as represented in Spearman’s famous g-factor model. In a conversation with biologists with whom he worked, Han had tried to explain what a factor model does and how it works statistically. Because they didn’t really get it, he had tried to use one of their own examples: lakes. Obviously, he said, the water quality in lakes is associated with various observables; for instance, the number of fish, the size of the algae population, the biodiversity of the lake’s ecosystem, the level of pollution, etc. Han explained that, in psychometrics, water quality would be thought of as a latent variable, which is measured through all of these indicators. He told me that the biologists had stared at him incredulously. No, they had answered, water quality is not really a latent variable. Rather, it is a description of a stable state of a complex system defined by the interactions between the observables in question. For instance, pollution can cause turbidity, which causes plants to die, which causes reduction of biodiversity, which allows the algae population to get out of hand, which increases turbidity, etc. (I am not a biologist, so if you really want to know what’s going on in shallow lakes, read this).

I vividly recall that Han sat in my room and said: couldn’t something like this be the case for general intelligence, too? That different cognitive attributes and processes, as measured by IQ-subtests (working memory, reading ability, general knowledge, etc.) influence each other’s development so as to create a positive manifold in the correlations between test scores? He drew arrows between boxes on a sheet of paper and held it up for me to see. I remember so well that I saw him do that. It was so simple but you could see that the ramifications were huge (although nobody at the time probably guessed just how huge). I said “I think that’s a really good idea”. He said: “Yes it is, isn’t it?!” and walked out with a smile. That drawing later became Figure 1b in Han’s mutualism model, eventually published in Psychological Review. The appendix to that paper formally proves that the mutualism model (which is basically a dynamical network model) can produce data that are exactly equivalent to the (hierarchical) factor model. That, I think, was the first real equivalence proof that was done in our group.

Essentially, this equivalence proof was what got the network approach going, because after many years of fruitless thinking about plausible causal mechanisms that would connect something like the g-factor to IQ or the internalizing-factor to insomnia, it suddenly appeared to us that networks could provide reasonable starting points for explaining correlation structures often observed in psychometrics in general, where the latent variable hypothesis provided very few believable stories. That’s why I decided to develop a general methodological framework around the idea that psychometric items can be profitably modeled using networks.

After starting our network research program in 2008, I played around with network models that we now know are in fact Ising models. I quickly found out that simulations from a network model for binary items produced data very close to IRT models, as would be expected from Peter Molenaar’s intuition and Van der Maas et al.’s proof. I gave several talks on this in various locations, include a keynote at a Rasch conference in 2010, where actual thunder broke after I said the Rasch model might be better thought of as a network (no coincidence, of course). However, I could never really show the formal equivalence between the network model and the latent variable model, partly because I lacked the mathematical skills and partly because I erroneously believed that I wasn’t simulating from an Ising model but from a closely related model.

That took a better mathematical mind: that of Gunter Maris, who was the first to really penetrate the nature of the correspondence between all of these models, and who, in a beautiful mathematical move, proved that they provide equivalent representations of the probability distribution of observables. I believe this happened in 2012, and consider this to be one of the main psychometric breakthroughs I have had the honor of witnessing. I expect it to have lasting effects on the psychometric landscape – we are merely at the beginning of exploiting the connection that this equivalence opens up: a secret tunnel that allows us to travel back and forth between a century of statistical physics literature and a century of psychometrics. The equivalence has now been written down in this chapter by Sacha Epskamp, which was written in 2014, in this paper by Maarten Marsman from around the same time, and of course, most recently, in the Kruis and Maris’ work. Also, Maarten Marsman has a forthcoming paper that extends the equivalence to a whole range of other models.

I have noted that some people think that, because there exists an equivalent latent variable model for each network model and vice versa, networks are equivalent to latent variables in general. This is erroneous. That one can always come up with some equivalent latent variable structure to match any network structure (and vice versa) doesn’t mean everything is equivalent to everything else. Care should be taken in distinguishing a statistical model, which describes the probability distribution of a given dataset gathered in a particular way, from the theoretical model that motivates it, which describes not only the probability distribution of this particular dataset, but also that of many other potential datasets that would, for instance, arise upon various manipulations of the system or upon different schemes of gathering data.

This all sounds highly theoretical and abstract, so it is useful to consider some examples. For instance, network structures that project from plausible latent variable models (e.g., scores on working memory items that really all depend on a common psychological process) can be (and as a rule are) highly implausible from a substantive theoretical viewpoint. Your ability to recall the digit span series “1,4,6,3,7,3,5” really doesn’t influence your ability to answer the series “9,3,6,5,7,2,4”; instead, both items depend on the same attribute, namely your memory capacity. This indicates that the theoretical model (both item scores depend causally on memory capacity) is more general, and thus different in meaning, from the statistical model (the joint probability distribution of the items can be factored as specified by an IRT model). Here, the statistical latent variable model is equivalent to a network model, but the theoretical model in terms of a common cause – memory capacity – is not.

Likewise, latent variable structures that project from networks can be highly implausible too. For example, an edge in a network between “losing one’s spouse” and “feelings of loneliness” can be statistically represented by a latent variable, as is true for any connected clique of observables. But from an explanatory standpoint, the associated explanation of the correlation in terms of a latent common cause makes no sense whatsoever. It is rather more likely that losing one’s spouse causes feelings of loneliness directly. Again, the difference in meaning between the theoretical model (losing one’s spouse causes feelings of loneliness) and the statistical model (the correlation between these variables remains nonzero after partialling out any other set of variables in the data) lies in the greater generality of the theoretical model, which extends to cases we haven’t observed (e.g., what would have happened if person i’s spouse had not deceased), cases in which we had used different observational protocols (e.g., observing the population register instead of administrating a questionnaire item on whether i’s spouse had deceased), or cases in which we would causally manipulate relevant variables. Statistical models by themselves do not allow for such generalizations (that is in fact one of the reasons that theories are so immensely useful).

Also, at the level of the complete model, the implications of latent variable models can differ greatly from those of network models. For instance, the model proposed for depression in Angélique Cramer’s recent paper is equivalent to some latent variable model, but not, as far as I know, to any of the latent variable models that have been proposed in the literature on depression. In general, if one has two competing theoretical models, one of which is a latent variable model with its structure fixed by theory, and the other of which is a network model with its structure also fixed, it will be possible to distinguish between these because the latent variable model equivalent to the postulated network model is not the same as the latent variable model suggested by the latent variable theory; Riet van Bork is currently working on ways to disentangle such cases.

Finally, even though latent variable models and network models may offer equivalent descriptions of a given dataset, they often predict very different behavior in a dynamic sense. For example, the network model for depression symptoms, in certain areas of the parameter space, behaves in a strongly nonlinear fashion (featuring sudden jumps between healthy and depressed states), while the most standard IRT model should show smooth and relatively continuous behavior. Relatedly, the network model predicts that sudden transitions between states should be announced by warning signals like an increase in the predictability of the system prior to the jump (critical slowing down, for which recent work has provided some preliminary evidence). There is no reason to expect that sort of thing to happen under the latent variable model that is equivalent to the network model, as estimated from the data, that was used to simulate from to get at these predictions.

So, the fact that two models are statistically equivalent with respect to a set of correlational data, does not render them equivalent in a general theoretical sense. One could say that while the data-models are equivalent, the theories that motivate them are not. This is why it is in fact not so difficult to come up with very simple data extensions that allow one to discriminate between observationally equivalent network and latent variable models. For example, in a latent variable model effects of external variables are naturally conceived of as going through the latent variable (e.g. genetic and environmental effects on phenotypic variation in behavioral traits, or life events that can trigger depressed episodes), whereas in network explanations these propagate through a network. This means that the latent variable model predicts the effect to be expressed proportional to the factor loadings (so the model should be measurement invariant over the levels of the external effect) while the network model predicts the external effect to propagate over the topology of the network (so the effect should be smaller on variables more distant from the place where the external effect impinges on the network).

Also, if experimental interventions are available, it should be reasonably easy to discriminate between latent variable models and network models, because in a latent variable model intervening on the observables is causally inefficient with respect to other observables in the model. This is because (in standard models) effects cannot travel from indicators to the latent variable, so they cannot propagate. However, in a network model, experimental manipulations of observables should be able to shake the system as a whole, insofar as the observables are causally connected in a network. So statistical equivalence under passive observation does not mean semantic or theoretical equivalence in general (see also Keith Markus’ interesting conclusion that (statistical) equivalence rarely implies theoretical (semantic) equivalence, but rather that statements to the effect that two models are statistically equivalent, as a rule, suggest that the models are not identical).

The post The meaning of model equivalence: Network models, latent variables, and the theoretical space in between appeared first on Psych Networks.

]]>The post How theoretically distinct mechanisms can generate identical observations appeared first on Psych Networks.

]]>The first sentence of the ‘About’ section on this website (and for that matter of most scientific publications about psychological networks) mentioned the increasing popularity of the network perspective in social sciences. Statements such as these essentially describe the increasingly popular practice among researchers to explain associations, observed between measured variables, as a consequence of mutualistic relations between these variables themselves.

Examining the structure of observed associations between measured variables is an integral part in many branches of science. At face value, associations inform about a possible relation between two variables, yet contain no information about the nature and directions of these relations. This is captured in the (infamous) phrase about the quantity measuring the extent of the interdependence of variable quantities: correlation does not imply causation. Making causal inferences from associations requires the specification of a mechanism that explains the emergence of the associations.

In our paper we discuss three of these, theoretically very distinct, mechanisms and their prototypical statistical models. These three mechanisms, represented within the context of depression, are;

The first mechanism represents the (until recently most dominant) perspective on psychopathology where a mental disorder is viewed as the common cause of its symptoms. The common cause mechanism is statistically represented by the latent variable model, and explains the emergence of observed associations through an unobserved variable (depression) acting as a common cause with respect to the observed variables (sleep disturbances, loss of energy, concentration problems). The manifest variables (symptoms) are thus independent indicators of the latent variable (mental disorder) and reflect its current state.

The network perspective on psychopathology is captured by, what we in our paper term, the reciprocal affect mechanism. In this framework the associations between observed variables are explained as a consequence of mutualistic relations between these variables. In this framework the unobservable variable depression does not exist, but is merely a word used to describe particular collective states of a set of interacting features.

The third, common effect, mechanism explains associations between observed variables as arising from (unknowingly) conditioning on a common effect of these variables, and is statistically represented by a collider model. In this framework the observed variables act as a collective cause towards an effect. An example of this is receiving a depression diagnosis (effect) as a consequence of the occurrence of multiple symptoms (causes) that are linked by the DSM to the term depression.

While each of these mechanisms proposes a radically different explanation for the emergence of associations between a set of manifest variables. We demonstrate in the paper that their associated statistical models for binary data are mathematically equivalent. From this follows that, each of these three mechanisms is capable of generating the exact same observations, and as such that any set of associations between variables that is sufficiently described by a statistical model in one framework, can be explained as emerging from the mechanism represented by any of the three theoretical frameworks.

Having multiple possible interpretations for the same model allows for more plausible explanations when it comes to the theoretical concepts and the causal inferences we obtain from the measurement model applied to our data. Furthermore, the historical success of theoretically very implausible models, such as the latent variable model can, in retrospect, arguably be explained by the equivalence of these three models.

However, it also means that obtaining a sufficient fit for the statistical models in one of these frameworks is by no means evidence that it is the mechanism from this framework that actually generated the observations. That is, there will always exist representations from the other mechanisms that can explain our observations equally well.

We should thus not only apply a network model to our data because it gives us a pretty picture (which it does), but because we believe that the associations between the variables we have measured are explained as a consequence of mutualistic relations between these variables themselves.

Abstract

Statistical models that analyse (pairwise) relations between variables encompass assumptions about the underlying mechanism that generated the associations in the observed data. In the present paper we demonstrate that three Ising model representations exist that, although each proposes a distinct theoretical explanation for the observed associations, are mathematically equivalent. This equivalence allows the researcher to interpret the results of one model in three different ways. We illustrate the ramifications of this by discussing concepts that are conceived as problematic in their traditional explanation, yet when interpreted in the context of another explanation make immediate sense.

— Kruis, J. and Maris, G. Three representations of the Ising model. Sci. Rep. 6, 34175; doi: 10.1038/srep34175 (2016).

The post How theoretically distinct mechanisms can generate identical observations appeared first on Psych Networks.

]]>The post Mplus 8.0 with Dynamic Structural Equation Models appeared first on Psych Networks.

]]>The network literature on psychopathology so far has been dominated by R, for instance when estimating cross-sectional network models via the packages *qgraph*, *IsingFit*, *mgm*, or *bootnet*, or time-series network models via *mlVAR* or *graphicalVAR*.

Modeling intensive time-series processes has become a topic important enough that the Mplus team asked Ellen Hamaker from Utrecht University to collaborate regarding the implementation of such models. I don’t know the details, but an invitation to a workshop states that Mplus will be able to estimate dynamical processes in data collected via “daily diaries, ecological momentary assessments (EMA), experience sampling methodology (ESM), and ambulatory assessments”. As usually, the Muthéns have found a sweet name and abbreviation for this: Dynamic Structural Equation Models (DSEM).

If you are interested in learning more about Mplus 8 that will come out this spring, there are several workshops planned. The first will take place July 13/14 2017 at Utrecht University, and I believe there will also be presentations by early adaptors of DSEM. If you are using Mplus 8 and want to present, email Rens van de Schoot.

Personally, I “grew up” with Mplus and it brings great benefits like fantastic state-of-the-art SEM modeling and very good support. Over the years, I have slowly transitioned to R because it is free open source software, deals much better with data management (with Mplus I usually need a second program to do the data management), and allows me to make my analyses 100% reproducible (everybody can get R and run my scripts, while not everybody can purchase Mplus and run my scripts). No matter how you stand on this debate, we should all be excited that Mplus is jumping on the dynamical process modeling bandwagon: this guarantees the thorough implementation of novel methodology, and the more software is out there that supports and further develops these models the better.

The post Mplus 8.0 with Dynamic Structural Equation Models appeared first on Psych Networks.

]]>The post Six new network papers, including a review of the empirical literature appeared first on Psych Networks.

]]>- Armour et al. 2016: “A Network Analysis of DSM-5 posttraumatic stress disorder symptoms and clinically relevant correlates in a national sample of U.S. military veterans” (dataset available)
- Cramer et al. 2016: “Major depression as a complex dynamical system”
- Fried et al. 2016: “Mental disorders as networks of problems: a review of recent insights”
- McNally et al. 2016: “Comorbid obsessive-compulsive disorder and depression: A Bayesian network approach”
- Schiepek et al. 2016: “Systemic Case Formulation, Individualized Process Monitoring, and State Dynamics in a Case of Dissociative Identity Disorder”
- van Nuijten et al. 2016: “Mental disorders as complex networks: An introduction and overview of a network approach to psychopathology”

The DSM-5 changed the PTSD symptoms substantially compared to the DSM-IV, and in this paper published in the *Journal of Anxiety Disorders* (PDF) we investigated the network structure of DSM-5 PTSD symptoms in a cross-sectional sample of 221 veterans with at least subthreshold levels of PTSD symptomatology. We also examined whether clinical covariates such as gender, anxiety, suicidal ideation, or quality of life are differentially related to the DSM-5 PTSD symptoms.

We published the dataset and syntax with the paper, so this is a nice start if you want to get into network modeling. Note that 221 people is a fairly small sample for modeling 20+ items in a network and that this paper definitely needs replication work. Below the two networks without and with covariates.

Read the full abstract here.Objective: Recent developments in psychometrics enable the application of network models to analyze psychological disorders, such as PTSD. Instead of understanding symptoms as indicators of an underlying common cause, this approach suggests symptoms co-occur in syndromes due to causal interactions. The current study has two goals: (1) examine the network structure among the 20 DSM-5 PTSD symptoms, and (2) incorporate clinically relevant variables to the network to investigate whether PTSD symptoms exhibit differential relationships with suicidal ideation, depression, anxiety, physical functioning/quality of life (QoL), mental functioning/QoL, age, and sex. Method: We utilized a nationally representative U.S. military veteran’s sample; and analyzed the data from a subsample of 221 veterans who reported clinically significant DSM-5 PTSD symptoms. Networks were estimated using state-of-the-art regularized partial correlation models. Data and code are published along with the paper. Results: The 20-item DSM-5 PTSD network revealed that symptoms were positively connected within the network. Especially strong connections emerged between nightmares and flashbacks; blame of self or others and negative trauma-related emotions, detachment and restricted affect; and hypervigilance and exaggerated startle response. The most central symptoms were negative trauma-related emotions, flashbacks, detachment, and physiological cue reactivity. Incorporation of clinically relevant covariates into the network revealed paths between self-destructive behavior and suicidal ideation; concentration difficulties and anxiety, depression, and mental QoL; and depression and restricted affect. Conclusion: These results demonstrate the utility of a network approach in modeling the structure of DSM-5 PTSD symptoms, and suggest differential associations between specific DSM-5 PTSD symptoms and clinical outcomes in trauma survivors. Implications of these results for informing the assessment and treatment of this disorder, are discussed.

— Armour, C., Fried, E. I., Deserno, M. K., Tsai, J., Southwick, S. M., & Pietrzak, R. H. (2016). A Network Analysis of DSM-5 posttraumatic stress disorder symptoms and clinically relevant correlates in a national sample of U.S. military veterans. *Journal of Anxiety Disorders, 45(44)*, 1–32. http://doi.org/10.1016/j.janxdis.2016.11.008 (PDF)

The new paper by Angélique Cramer and colleagues published in *PLOS ONE * (PDF) introduces both a conceptual and statistical model for major depression. The main contribution is that the authors show how individuals with more connected networks are more vulnerable to develop depression. These people can also transition from a healthy to a depressed state with only minimal amount of external stress (“cusp effect”), while a lot of stress reduction is needed to get that same system back into a healthy state (“hysteresis effect”). That is, if you stress a healthy system for people with highly interacting problems until it transitions into a clinical state, simply removing the stress may not be sufficient for the system to revert back. This is an exciting model that explains a number of observed phenomena in depression and now seeks further empirical validation in future studies.

In this paper, we characterize major depression (MD) as a complex dynamic system in which symptoms (e.g., insomnia and fatigue) are directly connected to one another in a network structure. We hypothesize that individuals can be characterized by their own network with unique architecture and resulting dynamics. With respect to architecture, we show that individuals vulnerable to developing MD are those with strong connections between symptoms: e.g., only one night of poor sleep suffices to make a particular person feel tired. Such vulnerable networks, when pushed by forces external to the system such as stress, are more likely to end up in a depressed state; whereas networks with weaker connections tend to remain in or return to a non-depressed state. We show this with a simulation in which we model the probability of a symptom becoming ‘active’ as a logistic function of the activity of its neighboring symptoms. Additionally, we show that this model potentially explains some well-known empirical phenomena such as spontaneous recovery as well as accommodates existing theories about the various subtypes of MD. To our knowledge, we offer the first intra-individual, symptom-based, process model with the potential to explain the pathogenesis and maintenance of major depression.

— Cramer, A. O. J., Borkulo, C. D. Van, Giltay, E. J., Han, L., Maas, J. Van Der, Kendler, K. S., … Borsboom, D. (2016). Major depression as a complex dynamical system. *Plos ONE. *http://doi.org/10.1371/journal.pone.0167490 (PDF)

We were invited by *Social Psychiatry and Psychiatric Epidemiology* to write a review on the empirical psychopathology network literature, which was a challenge because every time we wanted to submit the paper, a few new papers were out. The paper was published last week, is a bit more concise that we wanted it to be (the word limit was rather strict with 4500 words for a review), and focuses on the topics comorbidity, prediction of psychopathology onset or relapse, and clinical intervention. The paper is open access and available here.

Below an example of how comorbidity can be conceptualized from the perspective of the network approach: disorder X consists of the eight symptoms X1–X5 and B1–B3, disorder Y consists of the eight symptoms Y1–Y5 and B1–B3, and B1–B3 are bridge symptoms that feature in both diagnoses. In this case, a person first develops X3 in response to an environmental stressor E, then symptoms of disorder X, then bridge symptoms B, and finally symptoms of disorder Y.

Read the full abstract here.Purpose. The network perspective on psychopathology understands mental disorders as complex networks of interacting symptoms. Despite its recent debut, with conceptual foundations in 2008 and empirical foundations in 2010, the framework has received considerable attention and recognition in the last years. Methods This paper provides a review of all empirical network studies published between 2010 and 2016 and discusses them according to three main themes: comorbidity, prediction, and clinical intervention. Results Pertaining to comorbidity, the network approach provides a powerful new framework to explain why certain disorders may co-occur more often than others. For prediction, studies have consistently found that symptom networks of people with mental disorders show different characteristics than that of healthy individuals, and preliminary evidence suggests that networks of healthy people show early warning signals before shifting into disordered states. For intervention, centrality—a metric that measures how connected and clinically relevant a symptom is in a network—is the most commonly studied topic, and numerous studies have suggested that targeting the most central symptoms may offer novel therapeutic strategies. Conclusions We sketch future directions for the network approach pertaining to both clinical and methodological research, and conclude that network analysis has yielded important insights and may provide an important inroad towards personalized medicine by investigating the net- work structures of individual patients.

— Fried, E. I., van Borkulo, C. D., Cramer, A. O. J., Boschloo, L., Schoevers, R. A., & Borsboom, D. (2016). Mental disorders as networks of problems: a review of recent insights. *Social Psychiatry and Psychiatric Epidemiology, 1,* 1–32. http://doi.org/10.1017/CBO9781107415324.004 (PDF)

Richard McNally and colleagues have a new paper forthcoming in *Psychological Medicine* you should keep an eye out for! The paper isn’t online yet but I found it announced on a website, so I will limit myself to posting the abstract here as a teaser.

Background. Obsessive-compulsive disorder (OCD) is often comorbid with depression. Using the methods of network analysis, we computed two networks that disclose the potentially causal relations among symptoms of these two disorders in 408 adult patients with primary OCD and comorbid depression symptoms. Method. We examined the relation between the symptoms constituting these syndromes by computing a (regularized) partial correlation network via the graphical LASSO procedure, and a Directed Acyclic Graph (DAG) via a Bayesian hill-climbing algorithm. Results. The results suggest that the degree of interference and distress associated with obsessions, and the degree of interference associated with compulsions, are the chief drivers of comorbidity. Moreover, activation of the depression cluster appears to occur solely through distress associated with obsessions activating sadness – a key symptom that “bridges” the two syndromic clusters in the DAG. Conclusion. Bayesian analysis can expand the repertoire of network analytic approaches to psychopathology. We discuss clinical implications and limitations of our findings.

– McNally RJ, Mair P, Mugno BL, Riemann BC. Comorbid obsessive-compulsive disorder and depression: A Bayesian network approach. Psychological Medicine. Forthcoming.

Günter Schiepek and colleagues have published a case-report in *Frontiers in Psychology* (PDF). They estimated the network of cognitions, emotions, and behavior in a female patient diagnosed with borderline personality disorder and dissociative identity disorder. I admit that I’m not familiar with the modeling approach, but the paper features a larger number of graphs and I plan to devote some time next week trying to understand the model better.

Objective: The aim of this case report is to demonstrate the feasibility of a systemic procedure (synergetic process management) including modeling of the idiographic psychological system and continuous high-frequency monitoring of change dynamics in a case of dissociative identity disorder. The psychotherapy was realized in a day treatment center with a female client diagnosed with borderline personality disorder (BPD) and dissociative identity disorder.

Methods: A three hour long co-creative session at the beginning of the treatment period allowed for modeling the systemic network of the client’s dynamics of cognitions, emotions, and behavior. The components (variables) of this idiographic system model (ISM) were used to create items for an individualized process questionnaire for the client. The questionnaire was administered daily through an internet-based monitoring tool (Synergetic Navigation System, SNS), to capture the client’s individual change process continuously throughout the therapy and after-care period. The resulting time series were reflected by therapist and client in therapeutic feedback sessions.

Results: For the client it was important to see how the personality states dominating her daily life were represented by her idiographic system model and how the transitions between each state could be explained and understood by the activating and inhibiting relations between the cognitive-emotional components of that system. Continuous monitoring of her cognitions, emotions, and behavior via SNS allowed for identification of important triggers, dynamic patterns, and psychological mechanisms behind seemingly erratic state fluctuations. These insights enabled a change in management of the dynamics and an intensified trauma-focused therapy.

Conclusion: By making use of the systemic case formulation technique and subsequent daily online monitoring, client and therapist continuously refer to detailed visualizations of the mental and behavioral network and its dynamics (e.g., order transitions). Effects on self-related information processing, on identity development, and toward a more pronounced autonomy in life (instead of feeling helpless against the chaoticity of state dynamics) were evident in the presented case and documented by the monitoring system.

– Schiepek, G. K., Stöger-Schmidinger, B., Aichhorn, W., Schöller, H., & Aas, B. (2016). Systemic Case Formulation, Individualized Process Monitoring, and State Dynamics in a Case of Dissociative Identity Disorder. Frontiers in Psychology, 7(1545), 1–11. http://doi.org/10.3389/fpsyg.2016.01545 (PDF)

Update: I forgot to mention the new paper by van Nuijten et al. 2016 — so much work coming out it’s hard to keep up! The paper entitled “Mental disorders as complex networks: An introduction and overview of a network approach to psychopathology” (PDF) was published in *Clinical Neuropsychiatry*, and features an overview of the network approach. It makes for a great introduction and features multiple topics such as heterogeneity and comorbidity, and I see it as a concise and insightful summary of the fantastic Cramer & Borsboom (2013) that appeared in the *Annual review of clinical psychology*.

The paper took a while to get published, which may explain that there is only one reference to papers in 2016. Given the fast evolution of the field, some important developments are not mentioned. This caveat notwithstanding, a fantastic introductory read. Slate Star Codex wrote a blog about the network approach to psychopathology recently, featuring the paper by Nuijten et al. in more detail.

Read the full abstract here.Mental disorders have traditionally been conceptualized as latent variables, which impact observable symptomatology. Recent alternative approaches, however, view mental disorders as systems of mutually reinforcing symptoms, and utilize network models to analyze the structure of these symptom-symptom interactions. This paper gives an introduction to and overview of the network approach to psychopathology, as it has developed over the past years.

– Nuijten, M.B., Deserno, M. K., Cramer, A. O. J., & Borsboom, D. (2016). Mental disorders as complex networks: An introduction and overview of a network approach to psychopathology. *Clinical Neuropsychiatry, 13* (4/5), 68-76. (PDF)

The post Six new network papers, including a review of the empirical literature appeared first on Psych Networks.

]]>The post New paper: differential variability of variables drives network structure appeared first on Psych Networks.

]]>Differential variability — the phenomenon that items have drastically different variances such as positive affect and paranoia in the figure above — was brought up by one of the reviewers of my first published paper, and has stuck with me since. This is also the reason I mention this point in workshops, where I advise researchers to (1) always report both means and standard deviations of all their network variables in scientific papers, (2) and correlate the degree centrality of their nodes (the sum of absolute edges) with the standard deviation of their nodes. If the correlation is high, it means that differential variability of the nodes in the network may drive the centrality of the nodes (and hence the network *structure*).

The network below is an example. This is a symptom network estimated in a very large dataset (we’re submitting the paper soon so I won’t go into too much details). Node 2 (bottom right) is usually very highly connected with all other nodes for this disorder, but in our case, it’s only loosely connected. Why? Because the investigated population has extremely severe problems, and nearly everybody exhibits symptoms 2 in a very strong way. The variance of the item is much smaller than the variance of other items, which means it cannot exhibit strong connections with other items.

A new paper published in PLOS ONE entitled “Differences in Connection Strength between Mental Symptoms Might Be Explained by Differences in Variance: Reanalysis of Network Data Did Not Confirm Staging” by Terluin, de Boer and de Vet tackles this issue in more detail now.

They correctly describe the problem as follows:

*The strength of the connection between 2 variables (expressed as correlation or regression coefficient) rests principally on the amount of common (or shared) variance, relative to the total variance (i.e., common variance and unique variance including measurement error).However, the direct or indirect restriction of the variance of one or both variables (“range restriction”) reduces the connection strength [4]. The comparison of (sub)groups with different severity levels may result in different connection strengths between symptoms solely due to differences in variances [5]. Differential connection strength due to differences in variance is particularly a problem when psychological symptoms are studied in relatively healthy samples.*

The authors re-analyzed data from a paper by Wigman et al. (2013) who estimated temporal networks using linear regressions which is can be problematic when variables are not normally distributed. Instead, Terluin and colleagues used inverse Gaussian regressions that assumes positively skewed data and thus can take into account differential variability of items. Their results differ from those of Wigman et al. (2013):

*[our] alternative method failed to demonstrate increasing connection strength between mental states with increasing severity (i.e.,with increasing variance) in all models (except one) indicating that, when the skewness of the dependent variable was taken into account, no differential connection strength could be discerned across the severity subgroups.*

While the authors note some limitations of their study, they are definitely correct in stating that differential variability impacts on the network structure and thus the interpretation of findings. Wigman and colleagues also wrote a reply and essentially agree with the main point, too.

I hope somebody will pick up this topic for a larger simulation study that could be a project for a good student. It would be great to know to what extent this is relevant, and how similar the variance among nodes should be to guarantee that it does not bias the estimation of the network structure. The paper predominantly discusses time-series network models, and I’m curious to which degree this is an issue in cross-sectional models as well. In Gaussian Graphical Models, the state-of-the-art regularized partial correlation networks for cross-sectional data, we first estimate the correlation matrix among variables based on the polychoric correlations in case of skewed ordinal data. I’m aware of an unpublished simulation study showing that the polychoric correlations perform reasonably well in large samples in case of skewed data, and lead to weak false positive associations in case of extreme skew, but it would be good to explore this on more detail. Note that you can also apply the nonparanormal transformation before you estimate the network, which may reduce skew. Finally, I’m curious to what degree this biases the estimation of factor models. If anybody knows literature on the topic, would you be so kind and share it in the comments below?

Here is the full abstract of Terluin et al. 2016:

Background. The network approach to psychopathology conceives mental disorders as sets of symptoms causally impacting on each other. The strengths of the connections between symptoms are key elements in the description of those symptom networks. Typically, the connections are analysed as linear associations (i.e., correlations or regression coefficients). However, there is insufficient awareness of the fact that differences in variance may account for differences in connection strength. Differences in variance frequently occur when subgroups are based on skewed data. An illustrative example is a study published in PLoS One (2013;8(3):e59559) that aimed to test the hypothesis that the development of psychopathology through “staging” was characterized by increasing connection strength between mental states. Three mental states (negative affect, positive affect, and paranoia) were studied in severity subgroups of a general population sample. The connection strength was found to increase with increasing severity in six of nine models. However, the method used (linear mixed modelling) is not suitable for skewed data.

Methods. We reanalysed the data using inverse Gaussian generalized linear mixed modelling, a method suited for positively skewed data (such as symptoms in the general population).

Results. The distribution of positive affect was normal, but the distributions of negative affect and paranoia were heavily skewed. The variance of the skewed variables increased with increasing severity. Reanalysis of the data did not confirm increasing connection strength, except for one of nine models.

Conclusions. Reanalysis of the data did not provide convincing evidence in support of staging as characterized by increasing connection strength between mental states. Network researchers should be aware that differences in connection strength between symptoms may be caused by differences in variances, in which case they should not be interpreted as differences in impact of one symptom on another symptom.

Terluin B., de Boer, M. R., de Vet, H. C. W. (2016). Differences in Connection Strength between Mental Symptoms Might Be Explained by Differences in Variance: Reanalysis of Network Data Did Not Confirm Staging. *PLOS ONE 11(11): e0155205*. doi: 10.1371/journal.pone.0155205

I mentioned the paranormal transformation above in case your data is skewed. As Sacha Epskamp pointed out with the example below, the nonparanormal transformation does not do much for you in case of highly skewed data with only few categories:

```
set.seed(1337)
DF <- data.frame(
x = c(rep(0,90),rep(1,10),rep(2,10)))
hist(DF)
library("huge")
DFnpn <- huge.npn(DF)
hist(DFnpn)
```

Other options in this case are either to dichotomize your data and run an Ising Model, which should work unless you have very extreme skew, or estimate a Gaussian Graphical Model based on the polychoric correlations.

The post New paper: differential variability of variables drives network structure appeared first on Psych Networks.

]]>The post R tutorial: power issues & robustness of network models appeared first on Psych Networks.

]]>The authors estimate the network structure of 36 symptoms in 909 participants, and, as common in the literature, report (1) especially strong edges, (2) especially central symptoms, and (3) bridge symptoms that connect PTSD and MDD symptoms. In terms of sample size and power, this is a typical paper, and also similar to many of the papers presented at the ABCT conference last weekend that you can find here.

I want to use this paper as an opportunity to talk about power issues in network modeling, and show you how to estimate the robustness / accuracy / stability of the networks you obtain. I believe there is a real danger of an upcoming replicability crisis consistent with the rest of psychology, given that we are using highly data-driven network models,; below are some things you can do to look at your results in more detail.

This blog is a brief non-technical summary of a paper on robustness and replicability that is currently under revision at *Behavioral Research Methods* entitled “Estimating Psychological Networks and their Accuracy: A Tutorial Paper”.

Imagine we know the true network structure of a network with 8 nodes, and it looks like this (left side network, right side centrality values). All edges are equally strong, and that means that all centrality estimates are also equally strong.

And now imagine we take this true network, and simulate data from it for n=500. This gives us a dataset with 8 variables and 500 people, and now we estimate the network structure in this dataset, along with the centrality values.

As you can clearly see, neither edges nor centrality estimates are equally strong anymore, and if we were to write a paper on this dataset, we would (falsely) conclude that B-C is the strongest edge, and that B has the highest centrality. Because we simulated the data from a true model, we *know* all that all edges and centrality estimates are equally strong.

The issue is more pronounced with (a) fewer participants, (b) more nodes, or (c) both. The reason for this is that the current state-of-the-art network models we use in cross-sectional data, regularized partial correlation networks, require a very large amount of parameters to estimate.

I will to introduce two methods briefly that allow you to get to the bottom of this problem in your data.

First, we want to look into the question how accurate we estimate edge weights of a network. For this purpose, we take a freely available dataset (N=359), and estimate a regularized partial correlation network in 17 PTSD symptoms, which looks like this:

Using a single line of code in our new R-package *bootnet* to estimate **edge-weights accuracy**, you can now estimate the accuracy of the edge weights in the network, leading to the following plot:

On the y-axis you have all edges in the network (I omitted the labels of all edges here, otherwise the graph gets very convoluted), ordered from the highest edge (top) to the lowest edge (bottom). Red dots are the edge weights of the network, and the grey area indicates the 95% CI around the edge weights. The more power you have to estimate the network (the fewer nodes / the more participants), the more reliable your edges will be estimated, the smaller the CIs around your edges will be. In our case, the CIs of most edges overlap, which means that the visualization of our network above is quite misleading! While certain edges *look* stronger than some weaker edges, they are actually not different from each other because their 95% CIs overlap. It’s a bit like having a group of people with a weight of 70kg (CI 65-76kg), and another group with 75kg (CI 70-80kg): these 2 point estimates do not differ from each other because their 95% CIs overlap. In sum, with 17 nodes we would prefer to have many more participants than the 359 we have here.

If we now want to look whether specific edges are different from each other, we can run the **edge-weights different test** in *bootnet*:

Here you can see all edges on both x-axis and y-xis, and in the diagonal the value of the edge weights; black boxes indicate significant difference between 2 edges. After looking at the substantial overlap of CIs above, it is not surprising that most edges do not differ from each other. (Note that this test does not control for multiple testing; more about this in the manuscript).

The second big question we can try to answer is the stability of centrality. Let’s look at the centrality estimates of our network first, and focus on strength centrality, which is the absolute sum of edges that connect a given node to other nodes.

As you can see, node 17 has the highest strength centrality, but is the node actually more central than node 16 which has the second highest strength?

Instead of bootstrapping CIs around the centrality estimates (which are unbootstrappable, see supplementary of the paper), we look into the stability of centrality. The idea comes from Costenbader & Valente 2003 and is straightforward: we have a specific *order* of centrality estimates (17 is the most central one, then comes 16, then 3, etc), and now we delete people from our dataset, construct a new network, estimate centrality again, and do this many thousand times. If the order of centrality in our full dataset is very similar to the order of centrality in a dataset in which we dropped 50% of our participants, it means that the order of centrality is stable. If deleting only 10% of the our participants, on the other hands, leads to a fundamentally different order of centrality estimates (e.g. 17 is the least central now), this does not speak to the stability of centrality estimates.

You see that strength centrality (blue) is more stable than betweenness or closeness, which is what we have often found when using *bootnet* in numerous datasets now. In our case, the correlation between the order of strength centrality in our full dataset with a dataset in which we (2000 times) sampled 50% of the participants is above .75, which isn’t too bad. In the paper, we introduce a centrality stability coefficient which tells you more about the stability of your centrality estimates.

Finally, you can also check whether centrality estimates differ from each other, using the **centrality difference test**; node 17 is not more central than node 16. (Similar to the **edge-weights difference test**, this test does not control for multiple testing.)

- “I have a network paper and didn’t investigate the robustness of my network!” – Welcome to the club :). You can start doing it from now on!
- “Do I have to use
*bootnet*?” Hell no, if you have other / better ideas, you should definitely do that instead.*bootnet*is very new – the paper is not yet published – and there are probably many better ways to look into the robustness of network models. But it is a great start, and I’m very happy that Sacha Epskamp put so much work into developing the package and writing the paper about replicability with Denny Borsboom and me. For me personally, bootstrapping helped me to avoid drawing wrong conclusions from my data: if your most central node does not significantly differ from many other nodes in terms of centrality, you really don’t want to write up the symptom as being the most central, and if your visually strongest edge overlaps in CI with most other edges, you don’t want to conclude it is the most central edge. - “But Eiko, what about time-series networks?
*bootnet*only covers cross-sectional networks at present!” – Yes, we all need to write motivating emails full of beautiful and funny gifs to Sacha to shift his priorities! But seriously, we’re working on it. And please leave him alone, ok?

– Afzali, M. H., Sunderland, M., Teesson, M., Carragher, N., Mills, K., & Slade, T. (2016). A Network Approach to the Comorbidity between Posttraumatic Stress Disorder and Major Depressive Disorder: the Role of Overlapping Symptoms. *Journal of Affective Disorders.* http://doi.org/10.1016/j.jad.2016.10.037

Background. The role of symptom overlap between major depressive disorder and posttraumatic stress disorder in comorbidity between two disorders is unclear. The current study applied network analysis to map the structure of symptom associations between these disorders.

Methods. Data comes from a sample of 909 Australian adults with a lifetime history of trauma and depressive symptoms. Data analysis consisted of the construction of two comorbidity networks of PTSD/MDD with and without overlapping symptoms, identification of the bridging symptoms, and computation of the centrality measures.

Results. The prominent bridging role of four overlapping symptoms (i.e., sleep problems, irritability, concentration problems, and loss of interest) and five non-overlapping symptoms (i.e., feeling sad, feelings of guilt, psychomotor retardation, foreshortened future, and experiencing flashbacks) is highlighted.

Limitations. The current study uses DSM-IV criteria for PTSD and does not take into consideration significant changes made to PTSD criteria in DSM-5. Moreover, due to cross-sectional nature of the data, network estimates do not provide information on whether a symptom actively triggers other symptoms or whether a symptom mostly is triggered by other symptoms.

Conclusion. The results support the role of dysphoria-related symptoms in PTSD/MDD comorbidity. Moreover, Identification of central symptoms and bridge symptoms will provide useful targets for interventions that seek to intervene early in the development of comorbidity.

– Epskamp, S., Borsboom, D., Fried, E. I. (under revision). Estimating psychological networks and their accuracy: a tutorial paper. *Behavioral Research Methods*. (PDF).

The usage of psychological networks that conceptualize psychological behavior as a complex interplay of psychological and other components has gained increasing popularity in various fields of psychology. While prior publications have tackled the topics of estimating and interpreting such networks, little work has been conducted to check how accurate (i.e., prone to sampling variation) networks are estimated, and how stable (i.e., interpretation remains similar with less observations) inferences from the network structure (such as centrality indices) are. In this tutorial paper, we aim to introduce the reader to this field and tackle the problem of accuracy under sampling variation. We first introduce the current state-of-the-art of network estimation. Second, we provide a rationale why researchers should investigate the accuracy of psychological networks. Third, we describe how bootstrap routines can be used to (A) assess the accuracy of estimated network connections, (B) investigate the stability of centrality indices, and (C) test whether network connections and centrality estimates for different variables differ from each other. We introduce two novel statistical methods: for (B) the correlation stability coefficient, and for (C) the bootstrapped difference test for edge-weights and centrality indices. We conducted and present simulation studies to assess the performance of both methods. Finally, we developed the free R-package bootnet that allows for estimating psychological networks in a generalized framework in addition to the proposed bootstrap methods. We showcase bootnet in a tutorial, accompanied by R syntax, in which we analyze a dataset of 359 women with posttraumatic stress disorder available online.

The post R tutorial: power issues & robustness of network models appeared first on Psych Networks.

]]>