Estimating the predictability of questionable open-access journals

Can computational screening help assess questionable journals?

This study evaluates whether journal websites and publication metadata can support large-scale screening for questionable open-access journals. It treats automated predictions as a way to focus expert investigation.

What the study found

• Screening behavior can be adjusted to favor broader coverage or fewer false positives.
• Error analysis identifies cases such as discontinued titles, book series, and small society journals where classification is difficult.

How the study works

The work evaluates models using human-annotated reference sets and features describing website presentation, content, and publication metadata.

Scope and limitations

• A predicted label is not a final judgment about a journal or an individual article.
• Reference labels, incomplete metadata, and limited web presence affect the interpretation of model outputs.

Abstract

Questionable journals threaten global research integrity, yet manual vetting can be slow and inflexible. Here, we explore the potential of artificial intelligence (AI) to systematically identify such venues by analyzing website design, content, and publication metadata. Evaluated against extensive human-annotated datasets, our method achieves practical accuracy and uncovers previously overlooked indicators of journal legitimacy. By adjusting the decision threshold, our method can prioritize either comprehensive screening or precise, low-noise identification. At a balanced threshold, we flag over 1000 suspect journals, which collectively publish hundreds of thousands of articles, receive millions of citations, acknowledge funding from major agencies, and attract authors from developing countries. Error analysis reveals challenges involving discontinued titles, book series misclassified as journals, and small society outlets with limited online presence, which are issues addressable with improved data quality. Our findings demonstrate AI’s potential for scalable integrity checks, while also highlighting the need to pair automated triage with expert review.

Abstract from the original work, reproduced under its Creative Commons license. The overview above summarizes the study.

Cite this work

Han Zhuang, Lizhen Liang, Daniel E. Acuna (2025). Estimating the predictability of questionable open-access journals. Science Advances. https://doi.org/10.1126/sciadv.adt2792

@article{zhuang2025estimating,
  title = {Estimating the predictability of questionable open-access journals},
  author = {Zhuang, Han and Liang, Lizhen and Acuna, Daniel E.},
  year = {2025},
  publication_date = {2025-08-27},
  journal = {Science Advances},
  volume = {11},
  number = {35},
  pages = {eadt2792},
  doi = {10.1126/sciadv.adt2792},
  url = {https://doi.org/10.1126/sciadv.adt2792}
}