Graphical integrity issues in open access publications: detection and patterns of proportional ink violations

Do the areas in scientific bar charts match the quantities they represent?

This study examines violations of the proportional ink principle: the amount of visual ink representing a value should agree with that value. It develops an automated method for detecting these inconsistencies in scientific bar charts.

What the study found

• The detector achieved an AUC of 0.917 in the reported evaluation.
• The analysis estimated that about 5% of the bar charts in its sample contained proportional ink violations.

How the study works

A deep-learning method was applied to bar charts drawn from a collection of roughly 300,000 figures in open-access publications, followed by analyses of variation across fields and regions.

Scope and limitations

• The prevalence estimate describes the studied sample and chart type, not every figure in science.
• A graphical inconsistency does not establish why a chart was made that way or whether misconduct occurred.

Abstract

Academic graphs are essential for communicating complex scientific ideas and results. To ensure that these graphs truthfully reflect underlying data and relationships, visualization researchers have proposed several principles to guide the graph creation process. However, the extent of violations of these principles in academic publications is unknown. In this work, we develop a deep learning-based method to accurately measure violations of the proportional ink principle (AUC = 0.917), which states that the size of shaded areas in graphs should be consistent with their corresponding quantities. We apply our method to analyze a large sample of bar charts contained in 300K figures from open access publications. Our results estimate that 5% of bar charts contain proportional ink violations. Further analysis reveals that these graphical integrity issues are significantly more prevalent in some research fields, such as psychology and computer science, and some regions of the globe. Additionally, we find no temporal and seniority trends in violations. Finally, apart from openly releasing our large annotated dataset and method, we discuss how computational research integrity could be part of peer-review and the publication processes.

Abstract from the original work, reproduced under its Creative Commons license. The overview above summarizes the study.

Cite this work

Han Zhuang, Tzu-Yang Huang, Daniel Ernesto Acuna (2021). Graphical integrity issues in open access publications: detection and patterns of proportional ink violations. PloS Computational Biology. https://doi.org/10.1371/journal.pcbi.1009650

@article{zhuangacuna2021,
  title = {Graphical integrity issues in open access publications: detection and patterns of proportional ink violations},
  author = {Zhuang, Han and Huang, Tzu-Yang and Acuna, Daniel Ernesto},
  year = {2021},
  publication_date = {2021-12-13},
  journal = {PloS Computational Biology},
  doi = {10.1371/journal.pcbi.1009650},
  url = {https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1009650}
}