Dipper Technology's Zhao Jiehui: In the era of generative AI, how to truly integrate AI into the core production processes of enterprises? | WISE2024 King of Business
The environment is constantly changing, and the times are always evolving. The "Kings of Business" follow the tide of the times, insist on creation, and seek new kinetic energy. Based on the current situation of China's major economic transformation, the WISE2024 Kings of Business Conference will jointly discover the truly resilient "Kings of Business" and explore the "right things" in the Chinese business wave.
The environment is constantly changing, and the times are always evolving. The "Kings of Business" follow the tide of the times, insist on creation, and seek new kinetic energy. Based on the current situation of China's major economic transformation, the WISE2024 Kings of Business Conference will jointly discover the truly resilient "Kings of Business" and explore the "right things" in the Chinese business wave.
From November 28 to 29, the two-day 36Kr WISE2024 Kings of Business Conference was grandly held in Beijing. As an all-star event in the Chinese business field, the WISE Conference is already in its twelfth session this year, witnessing the resilience and potential of Chinese business in an ever-changing era.
2024 is a year that is somewhat ambiguous and with more changes than stability. Compared to the past decade, everyone's pace is slowing down, and development is more rational. 2024 is also a year to seek new economic momentum, and the new industrial changes have put forward higher requirements for the adaptability of each subject. This year's WISE Conference takes Hard But Right Thing as the theme. In 2024, what is the right thing has become a topic we want to discuss more.
It is 2024, and Chinese enterprises have experienced multiple stages from digitization, cloudification, to the current artificial intelligence revolution. However, these changes are inseparable from the most basic digitization - the digital construction of enterprises, which is equivalent to the "foundation" of these changes.
Founded in 2018, Deep Technology is a data intelligent infrastructure service provider that has been deeply engaged in the digital transformation of enterprises for many years and is committed to the landing of industrial generative AI and the construction of the basic platform.
At this year's WISE Conference, Chairman and CEO Zhao Jiehui brought the theme sharing of "The Landing of Industrial Generative AI and the Construction of the Basic Platform".
"Because of the emergence and application of generative AI, the original data platform market is expanding," Zhao Jiehui said. "All customers and suppliers in this industry need to reshape their product portfolios."
After the arrival of the generative AI wave, how to land large models has become a hot topic in 2024. Zhao Jiehui believes that it is far from enough for enterprises to only have the general and basic capabilities of large models. They must combine with smaller vertical models on this basis to form a complete technology stack in order to allow AI to enter the core production links.
Source: 36Kr
The following is the full speech of Zhao Jiehui, CEO of Deep Technology, sorted and edited by 36Kr:
Zhao Jiehui: Hello everyone!
Last year on this stage, I shared with you some initial thoughts on the integration of the enterprise service market and AI. After this year's development, we deeply feel that generative AI is completely reconstructing the enterprise service market. Today, I will continue to share some of our thoughts from this perspective.
Everyone must know that before this year, the enterprise service market has basically been doing data services. After this year's development, it will be found that in the construction of the digital platform, the market is being deeply reshaped by AI.
Here are a few key points to share.
First, the emergence and application of generative AI has made the original data platform market larger because AI can better exert the value of the entire data for enterprises;
Second, all customers and suppliers in this industry need to reshape their product portfolios, so that the upgrade of the data platform and the landing of customers are for the sole purpose of enabling the landing of AI. Because, the data platform and IT construction investment that are not integrated with AI lack certain practical value.
This morning, I was communicating with a customer. At that time, I mentioned one point: In the future, all data platforms, including the construction of data lakes and warehouses, are pointing to one goal: for the landing of AI in the industry.
This year, IDC has just released the "2023 China Manufacturing Big Data Solution Market Share" report. Currently, service providers that are more deeply integrated with AI have ranked higher, and Deep Technology has also significantly risen to the top few. Since its establishment in 2018, in the past six years, the entire product portfolio of Deep Technology has also undergone great changes with the development and landing of AI.
At the earliest, as you may know, in the enterprise service market, we made four products including the data middle platform, data integration tools, and lake warehouse engine, and finally unified and reconstructed them into the real-time intelligent lake warehouse platform FastData.
In the second half of 2022, based on the development of the model, the combination with data has emerged. Our products have been integrated with the domestic computing power platform to become an all-in-one solution, and finally formed a very basic platform that can land AI in enterprises.
Source: 36Kr
The current platform has been upgraded to the current architecture (as shown in the above figure). If an enterprise wants to land AI in any scenario, it must first have a computing power platform with a relatively cost-effective performance. After that, if the enterprise only does simple document abstraction work, it may not need a very complex integrated data platform. However, once it wants AI to go deep into the reasoning and decision-making process of the business itself, a very complete enterprise integrated data platform and model service platform are necessary.
The model service platform is not just a large language model. Because after the enterprise provides all the knowledge and data, to land a large model in the enterprise, two very big problems need to be solved.
The first problem is that a single large language model cannot solve this demand. Only with a large language model, some document abstraction and other work can be done. But if it is to assist in business decision-making and reasoning in the enterprise, it must be combined with data. FastData has made a deep upgrade for this goal.
The second problem is that only with a large language model, when the enterprise provides a bunch of drawings, documents, including very complex data, it is difficult to turn these into corpora and cannot be combined with in-depth professional mechanism models. In this process, a model is needed to form a complete model technology stack that serves the enterprise with the original vertical model or small model in order to complete the landing of the enterprise's large model.
Therefore, for the landing of enterprise AI, from the corpus engineering to the collaboration of multiple small models and large models to form a complete technology stack is a very important thing, and this is also the problem that our FastAGI large model service platform is to solve.
We are currently mainly focused on several important areas. The first is the AI-assisted quick response of the supply chain in the large consumer field. For example, for leading retail enterprises represented by Belle Fashion, we have made in-depth landing applications based on the original platform.
The second is in the production field, for the auxiliary design of drawings and the auxiliary adjustment of process parameters, we have also completed many landings.
In addition, we also have in-depth cooperation with medical institutions in Hong Kong to explore the landing of AI for healthcare.
Now, in the face of the core scenarios in large enterprises, from the supply chain to the production process, we have carried out in-depth cooperation with many industry-leading enterprises.
Generally speaking. For the enterprise to truly land the large model in the scenario, the integrated data platform is a very important prerequisite. In addition, in the model service, in addition to the large language model, it is also necessary to collaborate with multiple vertical models. For example, in the process aspect, the process compilation model and the original model need to form a complete technology stack to work.
On the matter of the basic large model, I have always had a view: In the landing of large enterprises, a large model can truly generate scene value. At present, we can see that the model with less than 72B parameters can cope with most application scenarios under the existing conditions.
If a large model is to be landed in To B customers and generate scene value, a certain level of model parameters is required, which will lead to a problem that the cost and the final benefit are not proportional. Therefore, whether it is the deep application of the supply chain or the deep application of the production process that we are currently doing, in the landing practice of the To B scenarios of large enterprises, the model of this scale can be mainly based on at present, mainly considering the comprehensive optimization of the effect and cost in the enterprise landing process.
We now have two basic models, 72B and 32B. Compared with the models for To C services, these two models have several important characteristics:
First, all the corpora frequently used by enterprises need to be expanded;
Second, in the enterprise application, there are three important capabilities that need to achieve 100% accuracy. The first is to count, and the ability to automatically generate accurate SQL needs to be extremely accurate. The second is to achieve 100% accuracy in the function call ability of various original systems. Also, the very complex drawings and documents, and the deep RAG need to be very accurate. In these aspects, Deep Technology has done very in-depth work.
Another is security, because information security will definitely be involved in the enterprise, and this aspect is also very important. We have jointly released the Chinese model security review model tool - Deepexi-guard with Southern University of Science and Technology to provide services for the model security of enterprises.
Back to the model service platform mentioned earlier, let's look at the relationship between this model service platform and the basic model. It can be found that when the model is really landed in the enterprise scenario, the large language model is the bottom layer. Then, the vertical models used and the large language model need to be well integrated and coordinated to be used in the enterprise and generate business value.
For example, if you want to optimize the process parameters, you need the original process mechanism model, the drawing design, and the vertical professional model to be well coordinated with the underlying large model to form the basic enterprise model technology stack.
When this thing is deployed at the customer, the customer will give you a large number of specification documents and knowledge, as well as a large number of system data connected to the customer. At this time, you need to have a very strong corpus engineering ability.
For example, after reaching a cooperation with a leading engineering design group, they gave us dozens of gigabytes of drawings and specifications. We need to build an enterprise-level knowledge base of integrated data by synthesizing the materials and data through multiple slicing rules and intelligent preprocessing corpora, and then fine-tune the model for special tasks to accurately understand the needs of the relevant questioners, thereby improving the answer accuracy. Therefore, model engineering is also very important.
When you think this model has been debugged, how to evaluate whether it can be put into work? Model evaluation becomes very important, and then above it is the application development platform that everyone knows.
In addition, we will face a big problem. Under the domestic computing power supply situation, how can a large number of enterprises quickly land this matter?
At present, we have launched the large model training and inference integrated server Fast5000E based on the mainstream chips at home and abroad.
In large enterprises, when actually landing a large model, the demand for basic training is not so great, mainly for SFT (Supervised Fine-Tuning).
A large amount of computing power demand is on the inference side. Currently, the ability level of domestic chips can basically support the enterprise's inference use. In addition, we have made an accelerator card, so that the available NVIDIA chips can run the model inference of a certain parameter after plugging in the accelerator card to complete the inference process. For many end-side needs, such as the production line needs to apply some inference devices, Deep Technology will subsequently launch some end-side inference products.
As for the landing of a large model in an enterprise, is it to rent the computing power of the computing power center, use the cloud, or build an all-in-one machine by itself?
My view is this. In the initial one to two years of construction, if there is a sufficient budget and the company can invest in the combination of the large model and the scenario, it is generally a large company. When such a company tries in the initial stage, it will tend to buy a training and inference all-in-one machine by itself, first build the scenario, and continuously improve it, which is very efficient.
When the landing of AI in the industry becomes more and more mature, and a large number of enterprises begin to do this, it may need to consider renting computing power more.
This is a very important process, and it changes with different customers and different stages. Therefore, if a large model is to be landed in the core scenario, at the beginning, the way of using the training and inference all-in-one machine can be used to accelerate the exploration process of the landing application of the large model in the core scenario of the enterprise.
Basically, after our software system is deployed in the enterprise, it can be used out of the box. Most customers can see the initial results within two weeks.
As mentioned earlier, if the enterprise only needs to organize meeting minutes and documents, it is basically no problem to use the basic model. However, if a lot of real-time business reasoning and decision-making assistance is to be done in this enterprise, then the upgrade of the data platform is very important.
The core here is that the data in the original system is actually the real-time status record of the business, but the logic, knowledge, documents, including the company's management norms, and how the documents should be managed, all have the contextual logic of the business.
The data platform that the current market needs is to have the ability to uniformly manage structured and unstructured data. Only in this way can the data platform form the real-time business status data of the enterprise, and be able to achieve the unified management of the structured and unstructured data of the business context logic. Form an enterprise super data fusion body to support in-depth reasoning after the model training is completed.
Briefly introduce our company. Since its establishment in 2018, Deep Technology has won many honors and has also been included in many authoritative lists in the AI field. At present, there are also many typical cases in the actual landing of AI in large enterprises. Everyone is welcome to visit and exchange.
Finally, to summarize, in the enterprise and the large industry to land artificial intelligence and large models, we need to think about two key issues:
First, upgrade the unified management of the structured and unstructured data platform of the entire enterprise to form a data basis for the deep business landing of AI.
Second, the enterprise needs a complete model service platform, which can integrate multiple vertical professional models based on the large language model to form a complete model technology stack, and can provide engineering such as corpus engineering, model security, and model evaluation.
Thank you all!