MAMORX: Multi-agent Multi-modal Scientific Review Generation with External Knowledge

How can automated reviews use figures and outside scholarly evidence?

MAMORX generates scientific reviews using multiple agents and multimodal foundation models. It considers manuscript text, figures, and citations, while retrieving external scholarly information to support assessments of novelty and related work.

What the study found

• In the reported arena comparison, judges preferred MAMORX reviews to the tested alternatives, including human reviews.
• Large context windows reduced the number of agents and processing time needed compared with the preceding approach examined by the authors.

How the study works

Structured outputs and function calling connect manuscript analysis with figure handling and external knowledge retrieval. Evaluation uses pairwise human preferences and Elo ratings on machine-learning and natural-language-processing papers.

Scope and limitations

• Preference for a review does not establish that every criticism or citation in it is correct.
• The evaluation's disciplinary scope and judging sample limit conclusions about other research areas or editorial decisions.

Using this work

The repository contains the review system and arena evaluation components. Check generated claims against the manuscript and cited sources. The repository specifies CC BY-NC-ND 4.0, with restrictions on commercial use and distribution of adaptations; consult its license before reuse.

Implementation and documentation

Cite this work

Pawin Taechoyotin, Guanchao Wang, Tong Zeng, Bradley Sides, Daniel E. Acuna (2024). MAMORX: Multi-agent Multi-modal Scientific Review Generation with External Knowledge. NeurIPS 2024 Workshop on Foundation Models for Science: Progress, Opportunities, and Challenges.

@inproceedings{taechoyotin2024mamorx,
  title = {MAMORX: Multi-agent Multi-modal Scientific Review Generation with External Knowledge},
  author = {Taechoyotin, Pawin and Wang, Guanchao and Zeng, Tong and Sides, Bradley and Acuna, Daniel E.},
  year = {2024},
  booktitle = {NeurIPS 2024 Workshop on Foundation Models for Science: Progress, Opportunities, and Challenges},
  url = {https://openreview.net/forum?id=frvkE8rCfX}
}