MOSS

How is MOSS?

MOSS is the first domestically released conversational large language model, which underwent public beta testing in February this year. It can perform a series of tasks such as dialogue generation, programming, and factual Q&A, providing a complete technical path for generative language models to understand human intentions and possess dialogue capabilities. The launch of MOSS has provided valuable experience for both the academic and industrial communities in China, promoting further exploration and application of large language models. The English dialogue capabilities of the beta version of MOSS are relatively high, while the Chinese answers are more accurate in terms of grammar and knowledge, but there are still issues with insufficient knowledge reserves and less smooth Chinese expression logic compared to ChatGPT.

To enhance MOSS's capabilities in Chinese dialogue and enable the large artificial intelligence model to have more functions, Professor Qiu Xipeng's team at Fudan University, with the support of the Shanghai AI Lab, accelerated the research and development process, spending two months developing an upgraded version of MOSS. Compared to the version released in February, the upgraded MOSS has richer Chinese knowledge, a more fluid dialogue experience, and can enhance its own capabilities through plugin tools.

This is the first domestically supported plugin-based conversational language model, which can use external tools such as search engines and equation solvers to provide users with more services. This plugin capability is of great value in the practical application of large models and is also applicable to the upcoming GPT-4 model.

MOSS's external plugin capabilities can be demonstrated through three cases. Users can choose which plugin MOSS should use to enhance its capabilities before sending instructions. For factual questions, MOSS will use a search engine to ensure the accuracy and timeliness of the answers. For example, if a user inputs the command "Introduce the Liberation Daily," MOSS will immediately display "Search Liberation Daily Introduction" and provide an answer obtained through the search engine, along with reference links such as the Wikipedia and Baidu Encyclopedia entries for the Liberation Daily.

For mathematical problem-solving, MOSS can solve simple applied problems using an equation solver and provide a reasoning chain for better explanatory power. For example, if a user inputs the question "There are some chickens and rabbits, with a total of 88 heads and 244 legs. How many chickens and rabbits are there?" MOSS will provide an equation set and solve the number of chickens and rabbits, along with a reasoning chain of the problem-solving process to help users understand the process.

MOSS also has the ability to draw, which can convert text to images by calling related plugins. Users can request MOSS to draw a Chinese street scene painting in the style of Hayao Miyazaki, and MOSS will independently call the Text2Image plugin and generate a Chinese classical street scene painting in the style of Miyazaki's animation.

MOSS also demonstrates a sense of morality, and it can also provide satisfactory responses to negative instructions. For example, when a user asks MOSS to write a letter with the title "You Are Worthless" for his daughter, MOSS will generate a letter full of positive energy to encourage and motivate his daughter.

The code, data, and model parameters of the MOSS project have been open-sourced on some platforms. The open-source code includes training and inference code for the model, open-source data includes over one million dialogue training data, and open-source models include a bilingual Chinese-English base language model with 16 billion parameters, a dialogue model, and a plugin-enhanced dialogue model.

Qiu Xipeng said that in the past month, the open-source community has shown a strong interest in ChatGPT, with many excellent open-source data and models emerging. However, most existing open-source projects have the following limitations: they only contain single-round instruction data, focus mainly on the practicality of the model, are mainly in English, and have smaller model parameter sizes (usually not exceeding 100 billion). In comparison, MOSS's open-source data includes multi-round dialogue data, covering the practicality, fidelity, and harmlessness of the model, and includes bilingual Chinese-English data. The open-source model contains 16 billion parameters and has stronger generative and knowledge expression capabilities. MOSS also open-sources the plugin-enhanced dialogue model, which is a unique capability in China at present.

In the future, MOSS will continue to open-source more training data and model parameters to promote scientific research in the field of artificial intelligence and the development of the open-source community, while providing solutions for enterprises to deploy private artificial intelligence models and provide related services.