The project consists of three work packages (WP) that establish the
scientific and technical capabilities to carry out four use case studies (UC).
Work packages (WP)
WP1
In order for GenAI-based tools to be implemented in the healthcare system, the capabilities of these tools must be able to evaluate e.g. in terms of accuracy, text fluency, errors made, and the criticality of the errors. Currently, there is no unified metrics system for AI generated natural text. In WP1, we systematically collect information on various evaluation possibilities based on a literature review and interviews, and we formulate our proposal for a recommended evaluation matrix for the healthcare system.
WP2
In the GenAID project, we focus specifically on language models, a form of generative AI that can process and generate natural text. In WP2, we explore various methods for selecting the most optimal models for healthcare applications, particularly for understanding and generating Finnish text. The evaluation of the models also considers factors such as model size, cost, and data security issues.
WP3
Utilizing GenAI in healthcare data management is heavily regulated and data management needs to comply with EU regulation of 2017/745 Medical Devices Regulation (MDR) and 2016/679 General Data Protection Regulation (GDPR) and National regulation of 552/2019 Secondary use of health social data. Various risks have been identified regarding the use of GenAI in processing patient information. Most important are data privacy, information quality and bias, transparency, sustainability, questions regarding ethics and accountability, interpretability as well as lack of domain specific LLMs in social and health care.
In WP3, we establish a setup within the HUS Acamedic secure environment, where UC 1-4 can be implemented using real healthcare data, but in a secure manner and in full compliance with all legal requirements.
Use case studies (UC)
UC1
The European Union has issued recommendations for cancer screening. This part of the European Union's broader cancer screening program includes a proposal according to which lung cancer screening should be offered to former smokers and current heavy smokers who are aged between 50 and 75. However, identifying former and current heavy smokers from health records is not straightforward since this data is often unstructured and ways to include it in the patient records vary. Manually identifying the patients is not feasible.
In UC1, we investigate whether language models (LM) can be theoretically used to identify risk factors and potentially select patients for intervention. This could offer future opportunities for more personalized risk factor screening, which is in line with the direction the EU has gradually been moving towards.
UC2
It is not uncommon for patients to have entries for thousands of patient record entries and finding relevant information from the medical records can be time consuming. Additionally, all relevant information may not necessarily be found. This can lead to a decline in the quality and equity of care, unnecessary investigations, and, in the worst case, jeopardize patient safety.
In UC2, we explore how effectively a language model (LM) can summarize essential patient information into a concise summary that would benefit the physician, particularly when they are aware of the patient's reason for visit.
UC3
The amount of medical information grows at a rapid pace and doubles in 73 days today. This makes it much tougher for physicians to follow all updated guidelines and other knowledge. GenAI has been recognized to be able to assist professionals in utilizing new information in patient care. In this use case, we investigate, for selected patient groups, the capability of GenAI to compare patient record entries with current clinical guidelines, thereby facilitating the work of physicians and improving the quality and equity of care.
In UC1-UC3, we utilize primary healthcare registry data.
UC4
The data used in UC 4 consist of approximately 100 patients recruited to the study. Their doctor visit has been recorded and the recording, its transcript and a professional's evaluation of the patient record entry generated by LLM will be analyzed. Our primary goal is to determine whether GenAI speeds up patient consultations and how much post-visit correction is required by the professional. UC4 uses consent-based prospective data.