How I use AI for occupational health literature reviews

AI-generated manga-style 'reporter' v 'detective' in a research library with late 1980s computers attached to modern LCD screens, paper falling from nonexistent fax machines and printers and what appear to be early videophones. Always check AI output!

AI has become a game changer for me when I’m reviewing medical and scientific literature to give advice and opinions to companies or organizations. If you ever find yourself having to rapidly review, absorb, understand and comment on a big tranche of written material to help you make decisions or recommendations, it might be a game changer for you too.

Here’s how I use an AI tool to help me pull relevant points out of a series of papers, find connections between them and interrogate the literature to get the information I need. I use it day-to-day when I’m giving strategic occupational health advice that’s based on the latest evidence, but the approach is a great part of the toolkit when writing a paper or conducting formal academic research.

The tool – NotebookLM

Google NotebookLM is an AI tool which lets you upload or link to several different sources and ask questions based on what’s in those sources. It will quote lines from those sources when answering questions, so you can cross-check for accuracy and ask further questions to gain deeper insight.

How to approach the problem – ‘reporter’ or a ‘detective’?

Being strategic about using the tool and your research strategy can make you much more effective. One way to research a topic is to find textbooks, papers, articles and so on about a subject, and read as much as possible. This ‘reporter’ approach can be useful when you are just starting out in a field you know little about. The idea with this approach is to start with an open mind and come to conclusions based on your evolving understanding of what you’ve read.

While it has its uses, it is extremely time consuming and can result in a never-ending rabbit warren of reference-chasing, while losing track of what you read and who said what. Preferentially remembering interesting, unusual or trivial points in your research odyssey can be an issue – a form of recall bias.

The alternative is a hypothetico-deductive or ‘detective’ approach - you think of a hypothesis and target your research to support or reject that hypothesis. This can be a much more efficient use of time and energy and encourages you to be expansive in your thinking as you’re exploring a topic. A disadvantage is that people doing this can be subject to errors like anchoring bias (where you prefer the first piece of evidence you explore) or conservatism bias (where you don’t revise your opinion when presented with new evidence to the contrary). A detective approach requires an ability to change your mind and be flexible when presented with new evidence.

Both have their place, and when you’re new to a subject, there’s no substitute for taking a ‘reporter’ approach and reading in some detail as you get your bearings. However, trying to absorb ever-more information gets harder as you get deeper into a subject. Changing to detective mode sooner rather than later can make the process much more effective.

Step 1: gathering material and loading it into the tool

Nothing revolutionary here: I use Google scholar, PubMed, JSTOR or any number of other resource compilations to find abstracts and papers relevant to what I’m researching. Depending on the project, I may stick entirely to peer-reviewed sources, or expand my search to grey literature, newspaper articles, blogs, YouTube videos and more.

I don’t just dump any vaguely relevant source into the tool. It’s important to triage everything that goes in, as if you put garbage information in, you’ll get answers based on it. There are ways to use AI to triage sources but most of the time I do this manually – skimming through everything, looking at where it was published, who wrote it, whether it makes sense, and whether I could justify basing a decision on it if I were challenged down the line. Keeping this ‘human in the loop’ step when setting up reduces the risk of AI hallucinations and lets me make sure I have overall control of what I want to review.

NotebookLM lets you upload pdf or other file types including audio, or link to files in a Google drive, YouTube links, website links or just copy text directly into the source manager.

Step 2: formulate questions and iterate

I try to be clear about my objectives early on, as it’s easy to get sidetracked when delving into several sources on an interesting topic. It’s worth putting time and thought into this as it provides clarity and focus as the project progresses. I make a point of noting down what I want to get out of the literature review and the key questions I want to answer. What do I want to learn? Am I writing a report for someone else, and if so, who is the audience?

If you’re struggling with questions to ask, you can get NotebookLM to help you. Use your objectives and tell the tool what you’re trying to accomplish, who your audience is, what the output of your project will look like, and ask it to suggest questions or lines of enquiry. It’s a great way to get your thought process rolling and build some momentum.

When you start your review, questions may help you understand the range of opinions in the materials you load into your notebook, the main thrust of arguments for and against a position, and whether there are any consensus views or outlying positions you’ve been asked to comment on.

As you get deeper into understanding the material you can use your questions to help you find links between disparate concepts and find out whether different sources contradict each other on particular points.

One killer application I’ve found for this tool is source-checking. Quoting or relying on a statement in a paper which itself is a citation of another paper can be risky because so much gets lost in translation from one citation to the next; a lot of citations are simply inaccurate. Checking a citation used to mean trawling through the cited paper manually, trying to parse what you were reading against what the citing paper claimed. It’s a terrible time-sink, right at the point of writing-up where you’re nearly ready to publish. Now I just load the cited paper into NotebookLM and directly ask if the paper says or implies what’s claimed in the citing paper. If it does, it’ll give me the original statement I can cite myself. If not, it’ll pick out points which might be tangentially related and I can decide if this citation is one to keep or discard.

Step 3 – get adversarial, check the sources and drive towards your conclusion

Let’s say you’re asked to give an evidence-based opinion on whether a treatment for back pain is effective. A stereotypical ‘reporter’ approach would see you read as many papers as possible, possibly look for some meta-analyses or Cochrane reviews, and see how the balance of opinion landed. A stereotypical ‘detective’ approach would see you decide right at the start that the treatment is the gold standard option (or a waste of time), load those papers into NotebookLM, and ask it pointed questions which aim to prove you wrong or prove you right. The best way to proceed is likely to be somewhere between these extremes.

The ‘detective’ approach I’m suggesting may appear somewhat aggressive – a more measured ‘scientific’ approach might start with a specific hypothesis such as “physiotherapy is effective for back pain” and then seek to support or refute this. The reason I start at the extremes of the argument and then work backwards towards the middle ground, is because I’m trying to understand the robustness of arguments across the spectrum. I find that starting with an adversarial framing can help reveal where evidence boundaries lie which in turn can be more useful than starting from assumed consensus. It doesn’t always work though and so the flexibility to change your mind and approach when zeroing in on a position, and the approach you use, is a critical part of examining the literature in this way.

Ask questions that seek evidence to support your viewpoint, then look for evidence in your sources that contradicts what you think or believe. Find those answers that contradict each other, and ask further questions to help you assign a weight to a position you want to test. Play around with your questioning style to see what you get out – you don’t need to be polite!

You may want to load more papers into NotebookLM as you get to understand the subject and your position more thoroughly. Refer frequently to the original objectives you had to remain on track. As the project and your position crystallises, you can move your questioning to test your final position. What are the holes in the recommendations you’re making? Are there suggestions you’re making that are not based on the sources you’ve loaded into the tool? Are there any other factors to consider when you’re making recommendations in this subject area?

Conclusion

Over 1.5 million biomedical and life sciences papers are collected in PubMed every year – and that’s just one database. It’s already almost impossible to have an in-depth understanding of all the literature around a biomedical topic, unless you’re an ultra-specialist. More generalist clinicians and researchers need a way to bridge the gap in knowledge and understanding between themselves and their super-specialist colleagues. Using AI to filter, analyse, validate and integrate research and knowledge is going to become an increasingly essential skill. Having a go with NotebookLM is a great place to start.

Q&A

How does this help me understand the subject more if the tool is doing all the reading?

If you let the tool do all the reading, your understanding of the subject isn’t likely to increase that much. Some reading, especially at the start, is very important. What NotebookLM lets you do is use a triaging ‘skimming’ read to sense-check your sources, then delegate some of the more painstaking in-depth reading. If you’re doing a PhD or publishing a paper, that in-depth reading isn’t avoidable. However, if you’re appraising options in a business environment or presenting data for a training session, the balance shifts. The balance of depth and breadth is something all researchers need to consider, and using AI in this way helps you extract more information from your sources in a targeted way. I find it enormously improves my ability to quickly appraise and give reliable opinions on fast-changing scientific topics.

Even if you are researching for a PhD, the ability to extract specific points from a large number of sources is a very valuable capability.

Aren’t you oversimplifying by classing all research as ‘reporter’ or ‘detective?’

Yes. Other approaches are available, this is just the mental map I use when using AI to help with my research. These two positions are extremes, and an effective review will use elements of both. Understanding a topic in depth will help you find more papers to explore, and to ask the right questions. But deep-reading every paper of a selection of 30 would take hours I don’t have. Being aware of different styles and applying them judiciously, makes you more effective as a researcher and able to tailor your output to the task at hand.

How do I know the tool hasn’t hallucinated a response?

There’s always a chance that an LLM-based tool will make up, or hallucinate, its responses to you. The advantage of NotebookLM is that you’re restricting its answers to the sources you provide. The tool gives you precise references to the source in the answers it provides, so as long as you check what’s being claimed in its answers, you retain control of the discussion and can double check the accuracy of the output the tool gives you. You’re responsible for the output that you put your name to so checking everything for accuracy is a key final step.