The AI Committee of the American Society for Indexing has released a white paper investigating the quality of LLM-generated book indexes, “LLM-Generated Book Indexes: Can They Replace Professionally Created Indexes?”

Here is the abstract:

“Book indexes allow readers to quickly access the information they seek. Necessary criteria for a book index include completeness (an index must guide the reader to all indexable information in the book), navigability (an index must guide the reader to subtopics and related topics), and accuracy (an index should not contain false or inaccurate information and should reflect the author’s terminology). We investigated whether artificial intelligence (AI) based on large language models (LLMs) is capable of providing indexes that meet these criteria and found those indexes fall far short. At this time, AI cannot replace professional book indexers and we are doubtful that it will be able to do so soon, or even at all.”

The full paper can be found here: ASI White Paper on LLM-Generated Book Indexes