AMIA NLP WG 2024

Introduction

The application of Natural Language Processing (NLP) in everyday life has rapidly progressed over the past decades. This progress has been enabled by the availability of tools, large language models (LLMs), and resources that could be shared, reused, and fine-tuned to support users' projects and collaborations. Today, recently evolved generative AI is being integrated into general search and workplace productivity tools. Applying NLP to the textual content of patient electronic health records (i.e., clinical text) is constrained by strict patient privacy and confidentiality laws and regulations. These domain-specific peculiarities for access and sharing of resources (e.g., annotated text corpora) and tools (e.g., trained machine learning algorithms) require creative solutions. Despite these privacy restrictions, many research teams have succeeded in developing novel biomedical and clinical NLP methods, creating and then sharing resources based on clinical text in a thoughtful and sustainable manner. Despite the legal specifics surrounding patient data, NLP-based technologies have permeated clinical and translational research. Our general objective with this pre-symposium is (1) to provide a platform for the next generation of biomedical NLP scientists to get focused feedback on their in-progress graduate work from a panel of senior academicians, (2) to provide career advice and resources for career development to address a need identified at our last AMIA NLP WG business meeting due to rapid marketplace changes, (3) to demonstrate and train the latest achievements, resources, and tools within the biomedical and clinical NLP community along with their reusability, portability, and interoperability with a particular focus on prompt engineering, (4) provide a forum for people to quickly present their work, providing awareness and opportunities for networking and collaboration within the AMIA NLP community, and (5) to discuss ethical issues related to the emerging NLP technologies, especially as they relate to the healthcare industry (and adjunct industries).

Program

Time	Title	Speaker
8:20 – 8:30am	Welcome to AMIA 2024 NLP Working Group Pre-symposium	Yanshan Wang, AMIA NLP WG Chair, University of Pittsburgh
Session 1 – NLP Applications in Clinical Decision Support and Patient Care Session Chair: Satya Sahoo, AMIA NLP WG Vice Chair, Case Western University
8:30 – 8:45am	Machine Learning Pipeline Flags Instances of Acute Respiratory Distress Syndrome from Electronic Health Records	Felix Morales, Northwestern University
8:45 – 9:00am	Interpretable Differential Diagnosis with Dual-Inference LLMs	Shuang Zhou, University of Minnesota
9:00 – 9:15am	Generative Large Language Models in Electronic Health Records for Patient Care Since 2023: A Systematic Review	Xinsong Du, Brigham and Women's Hospital and Harvard Medical School
9:15 – 9:30am	Improving postsurgical fall detection for older Americans using LLM-driven analysis of clinical narratives	Malvika Pillai, VA Palo Alto & Stanford School of Medicine
9:30 – 9:45am	Deep Learning for Detecting Cancer Recurrence and Metastasis in Radiology Imaging Reports	Lucas Liu, Fred Hutch Cancer Center
9:45 – 10:00am	LLMs Cannot Beat Traditional ML Models in Clinical Prediction Yet	Canyu Chen, Illinois Institute of Technology
10:00 – 10:30am	Coffee Break
Session 2 – Ethical and Methodological Considerations in Clinical and Biomedical NLP Session Chair: Rui Zhang, AMIA NLP WG Chair-Elect, University of Minnesota
10:30 – 10:45am	Exposing Vulnerabilities in Clinical Language Models	Avisha Das, Mayo Clinic Arizona
10:45 – 11:00am	Approximate Randomization Technique for Comparing Performance Between Demographic Subgroups	Paul Heider, Medical University of South Carolina
11:00 – 11:15am	Natural Language Processing for Assessing Transparency of Clinical Trial Publications	Joe Menke, University of Illinois Urbana-Champaign
11:15 – 11:30am	A Privacy-preserving Approach to Ingest Knowledge from Proprietary Web-based to Locally Run Models for Medical Progress Note Generation	Sarvesh Soni, U.S. National Library of Medicine
11:30 – 11:45am	Sex and Age Biases in ChatGPT-4 and LLaVA for Skin Disease Identification	Zhiyu Wan, Vanderbilt University Medical Center
11:45 – 12:00pm	Advancing Digital Health Equity and Inclusiveness: Leveraging Large Language Models to Accelerate Literature Review	Taylor Harrison, Mayo Clinic
12:00 – 12:15pm	Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy	Daniil Filienko, University of Washington Tacoma

Organization

Yanshan Wang (chair), PhD, University of Pittsburgh, Pittsburgh, PA

Rui Zhang (chair-elect), PhD, University of Minnesota, Minneapolis, MN

Joseph M. Plasek (Secretary), PhD, Mass General Brigham, Boston, MA

Satya Sahoo (vice chair), PhD, Case Western Reserve University, Cleveland, OH

Steve Bedrick, PhD, Oregon Health & Science University, Portland, OR

Kirk Roberts, PhD, University of Texas Health Science Center at Houston, Houston, TX

Özlem Uzuner, PhD, FACMI, George Mason University, Fairfax, Virginia

Meliha Yetişgen, PhD, University of Washington, Seattle, WA

Hongfang Liu, PhD, University of Texas Health Science Center at Houston, TX

Stephane Meystre, MD, PhD, University of Applied Sciences and Arts of Southern Switzerland (SUPSI), Lugano, Switzerland

Chunhua Weng, PhD, Columbia University, New York, NY

Hua Xu, PhD, Yale University, New Haven, CT

Mesoud Rouhizadeh, PhD, University of Florida, Gainesville, Florida

Yifan Peng, PhD, Weill Cornel Medicine, New York City, New York

Abeed Sarker, PhD, Emory University, Atlanta, Georgia

Ramakanth Kavuluru, University of Kentucky, Lexington, Kentucky

Submission and Review

Important Dates

Submission Deadline: August 28, 2024
Notification of Acceptance: September 10, 2024
Workshop: November 10, 2024

Graduate student consortium

Graduate students are invited to submit applications for a podium presentation of their graduate research work (in the biomedical and clinical NLP fields). The submission is suggested to include the following sections:

• Aims and Objectives - State the main objective(s) of your project.
• Justification for the Research Topic - Explain the motivations and significance for your project.
• Research Questions - Stating your research question is essential. This might be done in a list.
• Research Methodology - If you already have plans for your research methodology, explain them here. If you have not found an appropriate methodology yet, or wonder which one to choose, this is also the place to mention it. In this case, list the requirements your methodology should fulfill.
• Research Results to Date - You are not required to have results. But if you already have some, present them here.
• References – Any relevant citation.

State-of-the-art biomedical and clinical NLP

Researchers are encouraged to submit the most recent research studies (published, in press, or under development projects), tools, resources, events, and community shared tasks. The following sections are suggested:

• Methods/Tools/Resources/Events/Shared Tasks Description
• Justification of the Inclusion – Explain the relevance, interest, and value of the submission to NLP WG and its impact on medical informatics
• Summary/Outcome – A summary of the outcomes, such as participants in the event, experimental outcomes of methods, etc.
• References

Submission Guidelines

AMIA Template

Submission Link

Welcome to the AMIA 2024

NLP Working Group Pre-Symposium

Nov 10, 2024

San Francisco, CA