Driving Industry Evolution: Exploring the Impact of Generative AI on Sector Transformation

Our efforts are inspired by our unique co-innovation initiative with world-renowned partners from a few core sectors, including finance, manufacturing, energy, and so on. These collaborations have highlighted a substantial demand for advanced generative AI tailored to enhance core business operations. However, in our dialogues with strategic partners, we have identified crucial gaps in current generative AI capabilities versus the specific needs of industry applications. These include a too-narrow focus on human-like AI but not critical industry applications, limitations in processing complex and noisy data, and concerns about reliability in complex decision-making scenarios. Our research is crucial in addressing these limitations and amplifying the underappreciated potential of generative AI in high-value sectors. We are focusing on two main approaches: developing domain-specific foundation models that enhance analytical and predictive capabilities or enable interactive and controllable simulations and creating a versatile foundation-model-as-agent system for diverse industry decision-making tasks.

Our first project is transforming the way industrial data is analyzed and utilized. Facing diverse data formats like tabular, time series, and graph from various sectors, we are employing Generative Data Learning to enhance the large language model with strong ability to interpret and process diverse data formats by transforming them into a unified, instruction-oriented language. With training over this diverse sector data for [numerous]tasks, this approach enables more intuitive data analytics and predictions across various industries. Initial experiments on a typical classification and regression task over tabular data have shown that even a relatively small-scale model enhanced by our Generative Data Learning approach can outperform both general large language models and traditional models like tree ensembles, particularly in few-shot scenarios. This suggests the significant potential for a single-model solution with no extensive model training or fine-tuning in exploring industrial data intelligence maybe with only few-shot examples.

Our second project is exploring building foundation models over domain-specific data, and we focus on financial markets given its fundamental data is orders. We have developed a dual-level foundation model called Large Market Model that uses transformers on both the order sequence to model the market dynamics and the order-batch sequence to align the market trend with control signals. The performance of financial market simulations based on this Large Market Model has been very promising. They have excelled in forecasting market trends, simulating extreme scenarios for stress tests, and detecting market manipulations efficiently.

Our third project focuses on creating a decision-making agent through the knowledge-augmented generation and adaptive retrieval. This agent is essentially a trainable model that generates and extracts domain-specific knowledge, dynamically updating itself and retrieving most appropriate knowledge to handle changing environment. This adaptive approach is particularly useful in many industrycontrol applications, such as HVAC control with the goal of optimizing energy use while maintaining comfort. Deploying this agent into this scenario has shown it can outperform traditional reinforcement learning methods, saving significantly more energy, especially in unknown environments or when facing perturbations.

In summary, at MSR Asia, we are committed to advancing the development of generative AI to catalyze industry evolution through innovative research and partnership. We will soon be sharing more details about these projects through upcoming papers and open-source initiatives. We invite you, especially our industry partners, to stay tuned and join us in driving these transformative efforts forward. Thank you.