Updated: Oct 20
Written by Dr. Susanne Friese
This article begins with a short methodological introduction to within and across-case analysis and then shows how such an analysis can be implemented in ATLAS.ti. I will show how you can use the Code-Document Table, the Code Co-occurrence Analysis tools, global filters, and networks to facilitate such an analysis. The examples shown are based on a small sample project that serves illustrative purposes only.
In qualitative research, analysis is built from stories told by research participants, observations, reports, and the like and based on the identification of key aspects of the phenomenon under investigation (Ayres et al., 2003). These key aspects are referred to as categories or themes.
They can be identified a priori, or developed during the process of analysis (Coffey & Atkinson, 1996; Patton, 2002; Sandelowski, 1993). Categories or themes may be manifested across individuals or single documents or might apply to all participants or all data in the study.
At times qualitative researchers only produce a list of main themes (or categories and their sub-codes) and do not take the analysis further. Richards (1998) referred to this type of analysis as “garden path analysis” (p. 324). This means the list of themes found in the data and their variations are only described, but they are not related to each other.
A list of themes by itself, however, has little explanatory power. They contribute little to idiographic generalizations that are typical for qualitative research. They are only ingredients. In order for an analysis to be complete, the themes (or categories) need to be reintegrated so that it can be seen how they work together in an actual (or constructed) case (Ayres et al, 2003).
In statistical research, the analysis is based on variables and the use of measurements. Just as the list of variables and their distribution only constitute the raw material for statistical analysis, so does the list of themes in qualitative research.
If you want to take the analysis a step further, the identified key elements need to be related to each other. Within-case and across-case analyses are a means to do that. Through recontextualizing the data, a within-case analysis helps you to see how key aspects vary in individual cases; an across-case analysis allows you to develop a synthesis that captures the essence or the variation of experience across individuals to see patterns and commonalities. Thus, both approaches are necessary to capture the true nature of an experience, both through its parts and as a whole. Only then can individual experience be recognized in a generalized way (Ayres et al., 2003).
Within-Case and Across-Case Analysis in ATLAS.ti
Depending on how a case is defined in your data set, you have three possibilities to compare data within and across cases. If your case is on the document or document group level, you can use the Code-Document Table.
If you analyse focus group data and define each respondent as a case, you can use the special focus group tool to auto code all of the respondents and their characteristics. With such data, the cases are embedded within each document and you need to use the Code Co-occurrence Table for a within or across-case comparison. The Code Co-occurrence tool is explained in more detail in the following article: Co-occurrence Analysis with ATLAS.ti.
Another option is to use the Query Tool in combination with the scope option that allows you to focus your analysis on a particular case, or to compare data across cases.
In this article, you will learn how to conduct a within-case and across-case analysis if your cases are based on the document or document group level.
Using the Code-Document Table and Co-occurrence Analysis for a Within and Across-Case Comparison
The Code-Document Table allows you to compare the distribution of code frequencies between documents or groups of documents. You can see how often a code has been applied in a document or document group, plus you can access the data behind each of the numbers in the table to recontextualize them.
Being able to breakdown the codings of a code by individual cases or groups is often an eye-opener. When you are coding the data, you work linear through each document attaching all codes that apply. Using the table allows you to inspect the data for selected codes or code groups only; you can zoom in to an individual case level, or expand the analysis to an aggregated level represented by document groups.
To illustrate the various analysis steps, in the following, a sample project is used whose database consists of evaluations of the computer game Minecraft found online. Extracted are evaluations by parents who do not play the game, parents who have some experience playing Minecraft themselves, and others who play the game but are not parents.
Figure 1 shows the list of categories and sub-codes that were identified. You can see the age ranges for which the game is recommended, the perceived benefits and downsides of the game, whether it is recommended or not and tips for parents; the number of text segments that capture a positive, a positive but, or a negative evaluation; you find information about how the game is described, the features of the game, the various modes, and the online community. I could continue to describe the various categories in more detail, but this would mean I don’t take the analysis beyond the “garden path”.
To see how the various categories are connected, comparing the various groups of respondents, here: parents who play, parents who do not play, and players (other), is a useful next step. To create a Code-Document Table:
> In ATLAS.ti Win, select Analyze, and from there Code-Document Analysis.
> In ATLAS.ti Mac, select the Analysis menu, and from there Code-Document Analysis.
On the left-hand side, you see four selection lists to create the table. You can select items from the list of codes and code groups for the rows, and items from the list of documents or document groups for the columns of the table. Once you have created the table, you can also switch rows and columns.
Figure 2 shows a comparison of those three groups for the categories BENEFIT and DOWNSIDE. The table cells show the absolute frequencies of how often a code occurs within a document group The table coloring gives you immediate feedback on which aspects are mentioned most or least frequently by each group. If you click on a cell, you can read the data that is behind the number, and you can also see whether other codes have been applied. With a double-click on a quotation, you can inspect the data within the larger context of the document.
The table also gives you some information about how often a code occurs in the entire data set, the number of documents in a selected group, and the number of quotations in each group.
Given the above example, we see that there are 18 respondents in the ‘parents who do not play’ group, 7 respondents in the group ‘parents who play’, and 15 respondents in the group ‘players (other)’. The total number of quotations in the first group is 108, as compared to 63 and 74 in the two other groups. Thus, comparing absolute frequencies does not give you an adequate picture. Therefore, you can normalize the data. The reference point for normalization is the document with the highest number of quotations for codes in the table. In this example, it is the first group. This means that the number of quotations per code is multiplied by the ratio of the sum of all quotations of the reference group (here: 46) and the sum of all quotations of the respective other groups. In the above example, the ratios are 46/22 and 46/13.
> In the Windows version you find the normalization option under settings; in the Mac version under options.
As normalization mostly gives you numbers with decimal places, you may want to display relative frequencies in addition to absolute frequencies (see settings or options).
If the document or document groups are shown in the columns, activate row relative frequencies for an across-case comparison.
Normalization highlights that people who play themselves report more benefits and fewer downsides. The only downside that players from a non-parental perspective report is that the game might become boring over time. The benefits are shown in green and the downsides in red.
Overall, parents who do not play report more benefits than downsides. They mainly fear the social and emotional dangers for their children when playing online with strangers. Interestingly, parents who play themselves see less educational benefits in the game than the respondents from the two other groups. For them, the creative element of the game appears to be more important.
We actually are now cycling from an across-case comparison to a within-case comparison. Both, the table coloring and the Sankey diagram also give an indication of the within-case distribution of the selected codes.
> If you want a numerical breakdown of the within-case distribution, activate column relative frequencies. Optionally, you can deactivate absolute frequencies, as shown in Figure 6.
To look further into the similarities and differences, the necessary next step is to read the data that is behind each of the table cells. To do so, click on a cell and read through the quotations in the Quotation Reader that is displayed next to the table.
In the article Co-occurrence Analysis with ATLAS.ti, I already pointed out the importance of writing in the process of analyzing qualitative data. This can be done in ATLAS.ti memos.
When creating a table and looking at code distributions within and across cases and at the quotations that belong to each cell, open up a memo alongside and write down your observations. If you find interesting quotations, you can immediately link them to the memo or insert them via copy and paste.
When you begin writing, relations in your data will emerge – I would say – almost by themselves. Of course, it takes some analysis experience to be able to see what unfolds in front of your eyes. It is, however, simply a matter of doing it. I would like to re-iterate a quote I already referred to in my last article: You need to do analysis in order to understand analysis (Freeman, 2017).
In the process of writing, I realized that there might be some interrelations between the codes of the category BENEFIT. See the note in the memo above in Figure 7. This can be examined, by running a co-occurrence analysis. The results are shown below:
Within-Case and Across-Case Analysis by Means of Global Filter Setting
As can be seen in Figure 8 ‘buildings things’ was frequently mentioned together with ‘creative’ and ‘educational’ Whether this holds true when you compare the data across cases, can be further examined by setting global filters for each group.
> To set a global filter in ATLAS.ti 23, right-click on a document group in the list at the bottom right and select the option Set Global Filter. In older versions, you need to open the Document Group section in the Project Explorer on the left-hand side.
The Sankey diagrams below show you the results of examining the relationship between building things, being creative and educational across three different groups of respondents:
What we can see here is that the relationship between ‘building things – creative – educational’ only applies to the first group. The educational aspect does not figure in for parents who play themselves; they associate ‘buildings things’ more with the collaborative aspect of the game, skill development, and creativity. For other players the relationship is there but literally very thin.
Global filters can also be applied to the tables. Figure 11 shows an example. We have seen that the game is mostly perceived positively, but there are two respondents in the sample who are very negative.
As Ayres et al. (2003) emphasized, it is important to look at individual cases in order not to minimize or even lose their voice and the significance of the single perspective through the generalization of the overall pattern.
Figure 11 shows the results of reducing the data to two individual cases captured by the document group ‘Evaluation: negative’. These two respondents did not mention any benefits, only the downsides of the game describing it as a dangerous place, not suitable for children.
While coding the data, I noticed that the perspective that Minecraft is a dangerous place was not supported by other respondents and I connected their statements about the dangerous side of the game via hyperlinks.
Those can also be visualized via a network (Figure 13). In the network, we see responses from case 36, case 27, case 41, and case 23. Thus, through networks, we can also examine across-case comparisons, as shown here on the individual document level.
Step-by-step through cycling back and forth between individual cases, within-case and across-case comparisons, you can advance your analysis. This process was described by Tesch (1990) as hermeneutical spiral. The generation of meaning and, subsequently, the idiographic generalization is achieved through an iterative process of comparisons at all levels, and in all accounts.
Qualitative researchers typically collect data from multiple participants and contexts. When analyzing the data, the researcher must develop an interpretation that on the one hand reflects and is true to the experience of the individual or a case, and on the other hand applies also across all the various accounts that make up the data set. In this article I have shown how to leave the ‘garden path’ and how you can begin to see connections in your data, be it within or across groups, or within or across categories and themes. With the tools ATLAS.ti provides, you can easily iterate between information that is relevant to all respondents, to only a sub set in your data, or exclusive to only one or a few informants.
If you want to learn more about ATLAS.ti, visit Qualitative Research Training.
If you are looking for support: join the Qualitative Research Community.
I would like to point out that the examples shown in this article are based on a small sample project with a limited number of respondents. The absolute frequencies are small, and relative frequencies therefore exaggerate the significance of a code. The same applies to normalizing the data. You always need to evaluate within the context of your own data whether these options are meaningful or not.
Ayres, L., Kavanaugh, K. and Knafl, K.A. (2003). Within-Case and Across-Case Approaches to Qualitative Data Analysis. Qualitative Health Research, Vol. 13 No. 6, July 2003, 871-883.
Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data. Thousand Oaks, CA: Sage.
Freeman, Melissa (2017). Modes of Thinking for Qualitative Data Analysis. NY: Routledge.
Friese, S. (2019). Qualitative Data Analysis with ATLAS.ti. London: Sage.
Patton, M. Q. (2002). Qualitative evaluation and research methods (3rd ed.). Thousand Oaks, CA: Sage.
Richards, L. (1998). Closeness to data: The changing goals of qualitative data handling. Qualitative Health Research, 8, 319-328.
Sandelowski, M. (1993). Theory unmasked: The uses and guises of theory in qualitative research. Research in Nursing & Health, 16, 213-218.
Tesch, R. (1990). Qualitative research. New York: Falmer.