Nature Medicine: AI beat usual tests at guessing who lung-cancer immunotherapy helps

A top medical journal published a study saying an AI tool beat the usual hospital tests at guessing who with common lung cancer will be helped by drugs that train the immune system to attack the tumor.

Those immune-system drugs do not work for everyone, and today’s tests are rough. Nature Medicine now has a named multi-hospital AI study on the record — including an honest drop when the tool left its home hospitals.

The paper, free to read under CC BY 4.0, is titled “Clinical usability of an explainable AI decision support tool and evaluation of multimodal models in NSCLC.” NSCLC means non-small cell lung cancer, the most common kind. The scores below are the authors’.

Named teams include a national cancer institute in Milan (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano) and the University of Chicago lung-cancer group. Arsela Prelaj is the corresponding author — the journal’s listed contact.

The authors call the project I3LUNG (trial NCT05537922). They say it is the largest real-world study of this kind, with 2,396 patients. That is their claim in Nature Medicine.

The AI used more than one kind of data: clinical notes and blood tests (CB), CT scans, digital slides of tumor tissue (DP), and DNA. Combining those streams is what they mean by multimodal. They tried two mix methods, called MLEF and DLIF. How those mixers are wired is in Sources.

Using only notes and blood, they report an AUC up to 0.77 on patients the model had not seen. AUC is a 0-to-1 ranking score. 0.5 is a coin flip. 0.77 is useful, not perfect. At other hospitals (EXVAL) the score fell to 0.55–0.72. They say those patients were different.

On that TEST set, they say the AI beat the usual markers: PD-L1 (a protein already used to guess who these drugs help), ECOG performance status (how well the patient functions day to day), neutrophil-to-lymphocyte ratio (NLR, an inflammation blood ratio), lactate dehydrogenase (LDH, a blood enzyme), and the Lung Immune Prognostic Index (LIPI).

Lung specialists and non-specialist doctors improved their guesses when given the explainable notes-and-blood tool, per the paper.

Adding CT scans and tissue slides looked better in some runs. The authors say that extra benefit is still uncertain and did not show up on TEST or EXVAL.

A follow-up study — watching new patients going forward — is underway in more than 2,000 people. This is not approved treatment and not FDA clearance.