Biases

Biases in the Justinianic Plague app

Catalogs are not neutral representations of reality. Any catalog represents an act of standardizing, ordering, and interpreting a body of information. At the same time, catalogs are useful since they allow us to make sense of a messy reality through establishing categories - which are themselves heuristics that promote structure over nuance.

The Justinianic Plague application that is the focus of this site is ultimately another catalog. Despite the integral shortcomings of catalogs and the problems involved in using them, we decided to attempt to ameliorate some of their more significant issues. Digital technology - though hardly free of biases of its own - nonetheless permits the display and connection of complex data in ways that are impossible to do in more traditional print format.

The main objective of this app is to facilitate explorative analysis. This objective is different from most other catalogs that are closer to reference works that facilitate data accessibility.

The app makes certain assumptions and takes certain shortcuts. When we developed it, our rationale was to include features that improve the analysis even if they also include biases as long as we considered that the improvements outweighed the biases. As such, the app is experimental in its attempt to harness the benefits of digital technology while also remaining critically aware of the complex processes that shape the meanings it conveys to users.

The following is a non-exhaustive list of these in attempt to be clear about the biases that the app’s structure introduces to the user’s understanding of its results. This list is a work in progress, and we would be glad to include additional issues with the database.

  • Absences are not displayed. The plague app does not visualize the much more frequent silences in the many sources that do not report on the same outbreaks. In total, hundreds of sources that were written during the period of plague and describe historical events or cultural attitudes during it do not discuss it. The presented primary sources, therefore, represent the few dozen sources that note the presence of plague in any form and their inclusion in a catalog thus presents a strong bias towards plague.
  • Background maps. Any map chooses to prioritize certain aspects of space over others and no map is neutral. We decided to include several types of maps to allow users to select the one that would fit the analysis they are interested in. We call particular attention to the default ancient map (Pelagios), which we have taken from the Pelagios website. This map is a digital representation of the Barrington Atlas of the Roman World, which itself is a composite of geography and human features from different points in antiquity. At high zoom levels the map displays a certain configuration of provinces, while zooming in displays known Roman roads and eventually cities and smaller settlements. None of these were static in antiquity, and many of them were obsolete during Justinianic’s reign.
  • Geographical polygons. The app displays search results spatially on the central map. These are usually vague by necessity and correspond to the often brief descriptions in the primary sources or modern catalogs. None of the borders of the polygons drawn on the map should be accepted as authoritative. A general rule of thumb is that the smaller the area is the more precise the plague reference. We can probably trust most sources that describe outbreaks in particular cities to have affected the city and likely its immediate surroundings. Moving to the level of the province and beyond it becomes very easy to visually exaggerate the spatial spread of plague even based on slim evidence. For ease of reference we decided to keep the polygons on the map. The borders of each of these can be disputed and should be understood as shorthand outlines of the province’s size.  At the highest hierarchical level are general references to outbreaks in “the East” or “around the world”. Since any attempt to display these shorthands on a map would cover much or all of the map with polygons we decided to refer to them only in text format to keep a balance between representing the sources and not following their exaggerations. The critical user should examine the text of the source that refers to an outbreak in a particular polygon along with its context to better estimate the coverage. Moreover, although the polygons cover the map with a uniform color, comparative evidence has demonstrated time and again that in reality plague’s spatial coverage could vary greatly even between adjacent local areas due to environmental (e.g. micro-ecological differences) and social (settlement patterns, trade networks) reasons. As above, the critical user should examine the primary source before reaching any conclusion.
  • Certainty/Severity level. We established this measure as an easy way to convey the challenges in interfacing between the primary source and the scholarly interpretation of that epidemic case. Although the measure is subjective, and users might have other estimates of the certainty/severity of a particular epidemic, we still believe that visually differentiating between outbreaks of different certainty or severity is useful to avoid the implicit understanding that all cases of plague in the sources are somehow equal. As such, low certainty/severity levels signal our subjective skepticism that a particular outbreak caused major societal disruption in a particular case. This is usually associated with features such as the lack of details in the primary source, clear ideological reasons the primary source has to exaggerate the supposed effects, or a general lack of corresponding evidence. High certainty/severity levels signal the opposite - a well-documented outbreak that caused major societal disruption in a particular local context. Users who find this measure unhelpful can turn it off within the app.
  • Temporal range of plague. The frequent challenge in converting vague and ambiguous data from the primary sources to the digital database format is particularly sensitive when displaying numbers. The only such number in the plague app is the date of the plague outbreak, which includes multiple issues. First, sources that date plague outbreaks do it with different chronological systems that overlap only partially with our Gregorian CE system. Thus a year such as AM 6000 could refer to one of two CE years. In this case we include the outbreak in both complete CE years even though in reality it may have been an event that took one month or even less. Other sources include only vague dating, allowing readers to date a plague outbreak to a certain temporal interval, for example by events that come before and after in the text. This dating method is imprecise and can easily be corrupted through subsequent copying and editing, or even modern scholarship. The plague reference can also be placed in a particular point in the text to serve an ideological purpose. Finally, many sources do not include any internal dating for their outbreaks and are therefore dated over longer time intervals. Sometimes, catalogs “attach” such an outbreak to another outbreak that can be dated more easily. In attempt to date the different epidemic outbreaks in the app, we considered both the internal references of the text, and the interpretation of earlier scholars in their catalogs. Read more about disagreement between catalogs.
  • Catalogs. Catalogs themselves include multiple biases. When the plague app allows users to visualize the different catalogs’ understanding of where a plague outbreak hit, it necessarily also visualizes the catalog’s inherent biases, which derive from editorial decisions of imputed values of the different variables. For example, the spatial extent of plague does not necessarily have to reflect past realities: a plague outbreak in a vaguely defined “East” does not necessarily mean that plague ravaged through the entire Middle East or Eastern Mediterranean.

Users should also keep in mind the following broader biases:

  • Partial coverage. The very partial survival of literary and non-literary sources from late antiquity must have resulted in the loss of other contemporary accounts that described plague. The image of plague we have, therefore, is limited.
  • Imprecisions in the primary sources. Scholars have long noticed that some of the details the primary sources report are imprecise. This is most easily noticeable in numbers. For example, the death counts we have for Constantinople (estimated population of 500,000) for the Justinianic Plague over the sixth century include:
    • 542 CE outbreak: 300,000+ deaths (John of Ephesos) or 10,000+ deaths/day (Prokopios)
    • 573-574 CE outbreak: 3,000 deaths/day (Michael the Syrian)
    • 585-586 CE outbreak: 400,000 deaths (Agapios)
    • 599? CE outbreak: 3,180,000 deaths (Michael the Syrian) or 380,000 deaths (Chronicle of 1234)

Not only are these numbers clear exaggerations, but they are also implausible as they suggest no visible depopulation despite the multiple outbreaks over only a couple of generations.  

Other quantitative attempts to measure plague mortality include “half the survivors [of other natural disasters]” - up to 1 trillion deaths (Prokopios), 99.9% (John of Ephesos), and a third of humanity (an inscription).