No more need for Ctrl+F. This brand-new AI paper tool helps you easily find the exact content you are looking for from the vast sea of original literature.

I'm begging you all! I already have way too many papers to read through, so why are more automatic paper-writing tools still popping up nonstop!

Recently, GitHub has launched yet another AI academic paper generation tool —

Open Academic Paper Gen.

Don't swipe away just yet!

Apart from being able to automatically generate a full first draft with references, Open Academic Paper Gen has one more feature you absolutely need to pay attention to —

When full texts of the cited papers are accessible, it will search for supporting evidence from the full text (key point) to verify the cited viewpoints in your paper.

To be fair, this is extremely practical.

Most of the time, academic paper writing starts from a specific research topic, where you continuously sort out viewpoints, find supporting evidence, and adjust your own judgments based on existing literature.

With AI, it can help you expand your set of arguments, and also retrieve sources for your supporting evidence.

But the problem is, sometimes the viewpoints and papers involved are real, but once you put the "viewpoint (provided by AI)" back into the full text and check it against the full context, you will find that the original author probably did not mean that at all.

Therefore, to solve this problem, after obtaining the full text of the literature, Open Academic Paper Gen will further locate the relevant paragraphs to judge whether this paper can actually support the corresponding viewpoint.

If there is page number information available, it will also mark the corresponding PDF page number in the notes, making it convenient for users to go back and check later.

In other words, it does not only check "whether this paper exists", but also verifies "whether this statement is actually made in that paper".

A Multi-Agent Workflow for Generating First Drafts of Academic Papers

Overall, Open Academic Paper Gen is a multi-agent workflow for generating first drafts of academic papers, which splits the entire process from topic selection to export into nine stages:

Defining the research scope, retrieving literature, cleaning and screening, analyzing trends, finding research entry points, outlining, writing, verifying, and finally exporting.

Similar to other AI academic paper writing tools, you can input a research topic, for example, "RSI in the physical world".

It will break down this topic into specific research questions, then generate corresponding Chinese and English keywords to search for papers on OpenAlex, Crossref, Semantic Scholar and arXiv.

The retrieved literature will be further deduplicated, scored, and then screened for relevance by the model. For highly relevant papers, it will also continue to search along their references, as well as subsequent studies that cite these papers.

It is equivalent to grabbing one relevant paper first, then following the clues to find a batch of related works, and the idea is somewhat similar to Connected Papers in the past.

(You know what I mean if you ever used that tool).

Next, the collected literature will be organized into an evidence table, summarizing the research questions, methods, data, conclusions and limitations of each paper.

With these materials, the system will then analyze research trends, and propose possible research perspectives, research gaps and hypotheses.

There is also a very interesting section here called Innovation Diagnosis.

Open Academic Paper Gen will evaluate the entry point from the dimensions of research question, method, data and perspective, point out the closest existing work in the literature pool, and list possible questions that reviewers may raise.

(This is equivalent to letting AI check if someone else has already done the work before you think you have created something as groundbreaking as ResNet).

Specifically, these judgments will be made based on the literature retrieved in this process. However, if no related work is found in the literature pool, it does not mean that no one in the world has ever done relevant research~

Finally, after the research perspective is determined, the system will continue to generate an outline and draft content by chapters.

After the first draft is finished, it will enter the citation verification and simulated peer review stage, and finally you can export files in Markdown or LaTeX format, as well as reference formats such as RIS and BibTeX.

In terms of specific operation modes, under the default mode, Open Academic Paper Gen will pause at each main stage, waiting for you to review, edit and confirm before moving on to the next step.

If you want to test the capability of AI research, you can also choose the full-automatic mode to let it run through the whole process without manual intervention.

Finally, users who are ready to run the project also need to pay attention to the model configuration:

Currently the project supports OpenAI and Zhipu GLM, but several stages including argument verification, uncited statement check and simulated peer review still require an OpenAI API Key.

How are citations traced to specific sentences in the original text?

To be honest, automatically searching for literature, writing a first draft, and then letting AI review the draft is already a common combination in many academic paper writing projects.

Some of them also add citation verification and manual confirmation to minimize the possibility of the model making up content out of thin air.

Within this workflow, Open Academic Paper Gen further refines the inspection and modification of citations.

After all, equipping a first draft with a list of references is one thing, while proving that these references can actually support the content in the main text is another.

Specifically, it splits the verification into three rounds of checks.

The first round: Check if the cited reference can be found in the existing literature pool.

During the writing process, each citation mark must correspond to one of the collected papers.

If the model temporarily generates a non-existent citation identifier, the system will mark it out for easy positioning and modification.

The second round: Verify the identity of this cited paper.

For papers with DOIs, their information will be checked against Crossref records first; if no matching result is found, the system will then confirm the information on doi.org.

If information such as title, author, and publication year does not match, a corresponding warning will be triggered.

The third round: Check whether the original text actually supports the cited statement.

At this stage, the check truly moves from "literature metadata" to "literature content".

After obtaining the full text, the system will find the paragraph that best matches the cited argument from it, and send it together with the opening part of the paper to the model, to judge whether the original text supports the statement in the main draft.

If the full text cannot be obtained, the abstract or excerpt will be used instead. When page number data is available, the warning will also mark the corresponding PDF page number, making it convenient for users to go back and check.

Of course, if the model does not find sufficient evidence, that does not mean the statement has been proven wrong.

According to the processing logic of Open Academic Paper Gen, when there is insufficient evidence, the corresponding content will be marked as "Unable to Judge".

Literature with too little text to verify at all will be marked as "Unverified" instead of being directly approved.

Therefore, overall, compared with the "one-click finish writing the whole paper" feature, the idea of Open Academic Paper Gen that connects literature sorting, evidence positioning and self-check of citations is really quite smart.

Especially when you read the viewpoints and arguments written by AI and always feel that "it seems to make sense, but I can't trust it completely", being able to find the corresponding original text at least gives you a handle to go back and verify the content.

By the way, this project is developed by an independent developer Mao Shu. In addition to Open Academic Paper Gen, he has previously released multiple projects such as AI screenwriter and AI research tools.

References

[1]https://github.com/mmlong818/open-academic-paper-gen

This article is from the WeChat official account "QbitAI", Author: henry, published with authorization from 36Kr.