A large-scale screening saw 3,000 people "treated" in a week, and the AI ophthalmology large model tells a new story.

Using a "mobile phone to diagnose diseases", a scene that sounds rather sci-fi, has actually become a reality.

In early December 2024, Ablimit Aj, an ophthalmologist at the First People's Hospital in Kashgar Prefecture, Xinjiang, just participated in a large-scale fundus disease screening activity "led" by a "smartphone doctor".

In this activity, users only need to take pictures with their mobile phones to learn about their eye health, complete the filling of medical record symptoms, and receive a preliminary diagnosis from an AI doctor. Behind this "mobile phone doctor" is actually an ophthalmic AI large model auxiliary tool. In this way, within just one week, the screening activity reached more than 30,000 users.

In Kashgar Prefecture, Xinjiang, due to the strong ultraviolet radiation and long sunshine hours, cataract diseases are highly prevalent and are the leading cause of blindness in the local area. However, due to the relative shortage of ophthalmologists and the low awareness of many patients about eye diseases, they often do not think of going to the hospital for examination until the condition is very serious, resulting in the missed best treatment opportunity.

Looking at the entire grassroots areas in China, the problem in Kashgar may not be an isolated case. This is why eye diseases have always been a problem that the government and the medical industry attach great importance to in China. In 2022, the "National Eye Health Plan for the 14th Five-Year Plan (2021-2025)" issued by the National Health Commission emphasized the need to "strengthen the prevention and treatment of key eye diseases in key populations", such as the prevention and control of myopia in adolescents, the improvement of cataract treatment, and the improvement of the early diagnosis and treatment capabilities of eye diseases such as fundus diseases and glaucoma.

Nowadays, with the gradual deepening of the application of cutting-edge technologies such as artificial intelligence in the medical industry and the emergence of more new auxiliary diagnostic tools, what kind of progress will be made in the diagnosis and treatment of eye diseases?

When Eye Diseases Meet the Ophthalmic Large Model

The eyes are the windows to the soul. Conventional fundus examinations and the quick intervention of doctors in inquiries can not only play a role in the diagnosis and treatment of ophthalmic diseases - relevant data statistics show that nearly 80% of fundus diseases are actually preventable and controllable; in other departments such as endocrinology, fundus images can also become an important reference for diagnostic tools. For example, the relevant parameters such as blood vessels and optic nerves on the fundus are also of great value in diagnosing systemic diseases such as diabetes and hypertension.

However, in China, such an inquiry process has not been widely promoted.

The small number of people with the awareness of ophthalmic disease screening is certainly one reason, but more importantly, the resources and cost investment for hospitals to set up ophthalmology departments is huge, and the ratio of the number of ophthalmologists to patients in China is extremely disproportionate. According to the new "China Eye Health White Paper", the number of ophthalmologists in China is only 44,800, which means that there is only 1.6 ophthalmologists for every 50,000 people.

In the grassroots medical market in the lower levels, this phenomenon is particularly serious. In 2021, a research team from the Ophthalmology Center of Beijing Tongren Hospital Affiliated to Capital Medical University conducted a survey on the ophthalmology clinics of 13,400 secondary and tertiary hospitals across the country. It showed that after the new medical reform in 2009, the balance of medical resources between regions in China has indeed improved, but in the western remote areas such as Tibet, the development of ophthalmologist resources is still "significantly lagging behind other provinces".

However, the training period of doctors is very long. At a time when the shortage of human resources cannot be quickly compensated, what the industry needs more is an accurate, efficient, and widely promoted eye disease diagnosis and treatment tool to solve the problem of eye disease diagnosis and treatment in a more convenient way.

In the past two years, the emergence of AI large model technology can be said to be timely.

In fact, the characteristics of ophthalmic diseases that highly rely on image diagnosis and require a large number of doctors to read films determine that this is a discipline that is very suitable for artificial intelligence technology. This is also the reason why the field of ophthalmic diseases has been a highland that major enterprises are competing for since the first wave of "AI + Medical" a few years ago.

However, due to the limited technical level and the limited understanding of the medical scene, there are still some bottlenecks in the development of early ophthalmic AI diagnosis and treatment products, such as the lack of standardized and high-quality eye disease data sets; the algorithm model's mining of ophthalmic image data is relatively shallow, so AI technology cannot fully play the role in the screening and discovery of new diagnostic targets; or because the AI system is too dependent on image data and lacks multi-source information fusion, resulting in poor interpretability of the diagnosis results.

These defects eventually lead to insufficient screening efficiency and accuracy. Or, although doctors can conduct remote inquiries, due to the large number of patients, it is still impossible to complete high-quality comprehensive screenings in a short period of time.

But now, with the application and development of large language models, these problems are gradually being solved. For example, the screening in the hospital where Dr. Abulimit is located uses the ChatZOC domestic ophthalmic large language model jointly developed by the Zhongshan Ophthalmic Center of Sun Yat-sen University (hereinafter referred to as "Zhongshan Ophthalmic Center") and Huawei.

This is a domestically developed ophthalmic AI large model based on domestic Ascend computing power. In the field of ophthalmology, Zhongshan Ophthalmic Center is very authoritative in the industry. It is the only public ophthalmic hospital directly managed by the National Health Commission, with an annual outpatient volume of more than 1.2 million people, and has a deep accumulation of ophthalmic specialist data. The participation of Huawei is mainly to provide computing power support for the inquiry system, assist in the training and reasoning of the ChatZOC large model, so as to more efficiently process massive data, complete complex computing tasks, and improve the scientificity of knowledge Q&A.

At present, the system has been equipped with many advanced functions such as fundus image quality control and lesion risk assessment, and can interpret various examination results such as visual acuity, intraocular pressure, slit lamp photography, and fundus photography, helping residents detect potential eye diseases in a timely manner.

Doctors use the ChatZOC tool to screen for eye diseases

It is worth mentioning that since the research was launched in March 2023, to Huawei's participation in the joint research and development in December of the same year, and to the official release in June 2024, it took only more than one year to develop the ChatZOC to achieve its current functions.

Professor Lin Haotian, the director of Zhongshan Ophthalmic Center and the hospital director, mentioned that after Huawei began to participate in the research, in order to implement this large model, the two teams spent more than 180 days, overcome more than 100 technical problems, and iterated dozens of versions, ultimately making the "model's ability several times stronger". Currently, ChatZOC has supported nearly 1.5 million inquiries and can reduce the workload of examination interpretation by 50%.

In this process, the application of AI large models in the medical industry has finally become an "expert" in understanding eye diseases from a "generalist", as expected by the industry. The transformation of ophthalmic diagnosis and treatment may begin from this.

Diagnosis, Research, Teaching, ChatZOC Enters the "Daily"

"It's like an ophthalmologist in your pocket, and it's the kind that stores all the cutting-edge clinical knowledge of Zhongshan Ophthalmic Center," Dr. Abulimit commented on this new assistant, ChatZOC.

In remote areas, patients no longer need to travel a long distance or wait in line for a long time to see a doctor. Under the implementation mode of "pre-inquiry - photo diagnosis - post-inquiry - ophthalmology Q&A", patients can obtain real-time online inquiries from the professional ophthalmic digital AI doctor of ChatZOC through just one mobile phone.

Subsequently, the real doctor will quickly complete the outpatient service based on the auxiliary diagnosis report output by ChatZOC and promptly inform the patient of any abnormal conditions to ensure the continuity of subsequent medical treatment.

In this activity in Kashgar, the team from Zhongshan Ophthalmic Center and local doctors together completed the centralized eye disease screening of 2 hospitals, 3 schools, and 2 village and town residents, covering more than 30,000 people, of whom 3,000 completed the pre-inquiry; in terms of the screening effect, 35% of the participants were screened for dry eye, refractive errors, cataracts and other diseases, and dozens of people were subsequently referred to large hospitals for treatment.

The reason why such results can be achieved in the initial application is highly related to the "doctor's thinking" that ChatZOC possesses. Lin Zhenzhe, the person in charge of the Digital Intelligence Technology Research and Development and Application Department of Zhongshan Ophthalmic Center, gave an example to explain that many patients do not understand their own disease symptoms. If the model is to truly help patients, what needs to be done is to "guide them to describe their conditions".

Therefore, unlike the conventional AI model, the AI doctor of ChatZOC not only pays attention to the disease demands of patients, but also pays attention to the emotional needs of patients and other issues. In this process, what patients feel is like "communicating with a real doctor".

At the same time, not only for medical auxiliary diagnosis, after the process optimization is completed, the ChatZOC large model can also be applied in multiple scenarios such as teaching, scientific research, and the construction of digital hospitals.

For example, in the field of teaching, medical students can interact with ChatZOC to simulate the doctor-patient Q&A process and cultivate the diagnostic and treatment thinking; in the field of scientific research, ChatZOC can also be used as a scientific research intelligent assistant to provide researchers with "one-stop scientific research project and tool management" to promote interdisciplinary integration and innovation.

In addition, with the promotion at the policy level, the construction of digital hospitals has become the mainstream direction of hospital development in recent years. In this regard, ChatZOC can also be connected to the hospital platform to provide patients with all-day eye health consultation. The existing test data shows that with the same manpower, the service efficiency of patients under the ChatZOC-assisted mode can be increased by 75%.

The ability to achieve such application effects is mainly due to the team's in-depth understanding of the medical scene during the development of the large model. An expert in the relevant field of Huawei said that when developing a medical large model, the intellectual property rights of the medical industry must be respected, "enabling the industry through AI and enabling industry talents to understand AI".

"Huawei is committed to doing a good job in the three bases of the industry: the solution base, the ecological base, and the talent base, to support Zhongshan Ophthalmic Center of Sun Yat-sen University and empower the medical industry with AI."

Helping Graded Diagnosis and Treatment, the Large Model Enlarges the "Inclusive Value" of Medical Care

In the medical industry, there is a famous "impossible triangle" theory: that is, in the traditional medical productivity, it is difficult to simultaneously take into account the three key elements of "quality", "cost" and "efficiency". And what they correspond to are actually the three core issues of "getting good treatment", "affording treatment" and "having access to treatment".

Nowadays, although AI large models cannot directly complete the entire process of seeing a doctor, they have become an important fulcrum for achieving the "impossible triangle".

For a long time, the balance of medical resources between regions has been the pursuit of the industry. In the past, the implementation purpose of "initial diagnosis at the grassroots level and transfer for serious diseases" in the three-level diagnosis and treatment system is precisely this. However, the traditional method relying on human resources is still difficult to solve problems such as differences in diagnosis and treatment levels and different hospital grading standards, resulting in patients' distrust of grassroots medical levels and still wanting to "gather" to see a doctor in top tertiary hospitals in big cities.

For example, in ophthalmic diagnosis and treatment, the research of the Ophthalmology Center of Beijing Tongren Hospital Affiliated to Capital Medical University mentioned earlier once pointed out that currently, the ophthalmic service pressure in the eastern coastal provinces in China is still significantly higher than that in the western region, and there is an urgent need to "strengthen the construction of ophthalmic service capabilities in less developed provinces so that residents can receive treatment locally".

Nowadays, with the emergence of large models, a new type of three-level diagnosis and treatment model led by technology is slowly brewing, and this phenomenon is expected to be improved.

Relying on underlying intelligent technologies such as AI, big data, and 5G, the latest feature of the three-level diagnosis and treatment led by AI large models is that patients can complete basic disease screenings "without leaving home" and "seamlessly connect to the medical services of top experts". At the same time, in this process, the diagnostic opinions given by the large model based on its independent learning ability will also provide a preliminary reference for doctors, thereby improving work efficiency.

Being able to "get good treatment" at the grassroots level not only builds a trust foundation for patients in the three-level diagnosis and treatment model, but also truly achieves the original intention of efficient utilization of medical resources.

ChatZOC screening results

After several months of attempts, ChatZOC has indeed made the field of ophthalmic diagnosis and treatment the first to become the "beneficiary" of the large model.

At present, Zhongshan Ophthalmic Center has jointly with many units across the country to build a "inverted pyramid" three-level diagnosis and treatment system based on the intelligent prevention, screening, diagnosis and treatment of eye diseases, which can cover the entire process of eye disease screening, diagnosis, treatment, follow-up and prediction: First, patients can conduct initial home-based intelligent eye disease screenings through their mobile phone terminals and obtain common eye disease prevention and nursing knowledge popularization;

In the second level, the grassroots medical units in this system will provide patients with corresponding intelligent diagnosis services;

If there are really intractable diseases that cannot be solved, in the third level, Zhongshan Ophthalmic Center will jointly with other top ophthalmic hospitals to provide expert-level diagnosis and treatment services.

Since this year, Zhongshan Ophthalmic Center has launched the "Brightness Journey in Hundred Cities" activity in collaboration with 136 medical institutions across the country, carrying out eye health screenings, free clinics and other activities in 108 cities and villages in 28 provinces/autonomous regions including Xinjiang and Tibet, serving nearly 150,000 people in total and screening out 100,000 patients with diseases such as cataracts and refractive errors.

"We expect that using the ChatZOC ophthalmic large model technology, more than 1 million eye disease patients can be served in the future."

This article is from the WeChat public account "36Kr", and 36Kr is authorized to publish it.