Shusheng General Large Model

Shusheng General Large Model: Open-world understanding and cross-modal generation by Shanghai AI Lab

Shusheng General Large Model Official Website, Shanghai AI Lab, Open World Understanding, Cross-modal Generation, Multimodal Interaction

Introduction to Shusheng General Large Model

Just as the human "five senses" are interconnected and inseparable, the boundaries between artificial intelligence's visual, linguistic, and audio modalities are increasingly merging. With the rapid development of artificial intelligence's perception, interaction, and generation capabilities, multimodal large models are propelling artificial intelligence into the "synesthesia" era.The Shanghai AI Lab (Shanghai AI Lab) recently released the Shusheng General Large Model system, including the Shusheng Multimodal Large Model (hereinafter referred to as "Shusheng·Multimodal"; InternLMM, Intern Large Multimodal Model), which is a large model with real-world understanding capabilities, enabling cross-modal generation and interaction. Shusheng·Multimodal contains 20 billion parameters, trained by 8 billion massive multimodal samples, supporting the recognition and understanding of 3.5 million semantic tags, covering common categories and concepts in the open world, and leading internationally in over 80 multimodal and visual tasks.

Shusheng General Large Model Official Website:https://intern-ai.org.cn/home

Main Functions

Currently, Shusheng·Multimodal has three core capabilities: open world understanding, cross-modal generation, and multimodal interaction.

Achieving world-leading levels in multiple multimodal and visual tasks

Core Capability One: Open World Understanding

In the research of artificial intelligence, "open world" refers to the real world that is not predefined, academic, or closed-set defined.

In traditional research, artificial intelligence can only complete predefined tasks, i.e., tasks defined by academic or closed sets, but there is a significant gap between the scope of these tasks and the real open world. For example, the ImageNet-1K academic set contains 1000 types of objects, of which there are about 2 types of flowers, 48 types of birds, and 21 types of fish; while in the real world, the number of flowers, birds, and fish is approximately 450,000, 10,000, and 20,000 respectively.

In the open world, Shusheng·Multimodal is continuously learning to obtain perception and cognitive abilities closer to humans.

In terms of semantic openness, Shusheng·Multimodal can recognize and understand over 3.5 million semantics in the open world, covering common categories of objects, object actions, and optical characters in daily life. Shusheng·Multimodal has transformed from solving predefined tasks to executing open tasks, providing strong support for the research of future multimodal AGI models.

In terms of task openness, Shusheng·Multimodal can treat images as a new language, allowing users to use natural language instructions to flexibly define and manage any visual tasks. This large model also has various levels of customizable visual perception, understanding, and logical reasoning capabilities, covering modules such as target objects, output formats, and task content.

Core Capability Two: Cross-modal Generation

Through joint learning, Shusheng·Multimodal can achieve inter-modal transformation.

Researchers tried to have Shusheng·Multimodal create a seven-character quatrain based on Zhang Daqian's "Summer Landscape on Lake and Mountain." The verification results show that after joint learning, Shusheng·Multimodal has already possessed good cross-modal generation capabilities from images to text and has accumulated a profound understanding of Chinese culture.

While generating text, Shusheng·Multimodal also provided creative ideas: determining that the image depicts a tranquil natural landscape; finding elements that can express the poet's thoughts and emotions, such as towering mountains, swirling clouds, and pine trees painted in the scene; composing verses based on these elements; and finally refining them according to the rhythm and prosody. Shusheng·Multimodal also particularly described the inspiration for the fourth line: drawing inspiration from the famous line by Tang Dynasty poet Wei Zhuang, "The spring water is as green as the sky, and the painted boat listens to the rain while sleeping."

Seven-character quatrain created by Shusheng·Multimodal based on Zhang Daqian's "Summer Landscape on Lake and Mountain"

Core Capability Three: Multimodal Interaction

Shusheng·Multimodal inherits the performance of the previous generation Shusheng in conventional predefined visual tasks while innovating human-computer interaction methods. Users can accurately convey instructions through cursor clicks, chats, and other means. Shusheng·Multimodal lowers the threshold for AI tasks, making AI a production tool accessible to the masses.

Multimodal understanding, generation, and interaction capabilities are becoming an important direction for the new round of evolution of large models. Looking to the future, Shusheng·Multimodal will continue to enhance model capabilities through original innovation, empower the innovation ecosystem through open source and openness, and better serve humanity with large models.