ChatGLM

ChatGLM

ChatGLM,Tsinghua technology,Zhipu AI's open-source GLM series dialogue model

Visit Site →

Official website of ChatGLM, Tsinghua Technology, GLM series dialogue model open-sourced by Zhipu AI

What is ChatGLM?

ChatGLM isZhipu AIcompany's open-sourced dialogue model series based on the transformation of Tsinghua University's technological achievements, supporting both Chinese and English languages. Currently, this series has open-sourced a model with 6.2 billion parameters. These models have optimized the architecture based on the advantages of GLM, reducing the threshold for deployment and application, enabling large-scale models to be applied on consumer-grade GPUs.

https://xinghuo.xfyun.cn/sparkapiViewFrom the perspective of technical route, ChatGLM has realized reinforcement learning of human alignment strategy, making the generated dialogue more in line with human values. Its current capabilities mainly include self-cognition, outline writing, copywriting, email writing assistant, information extraction, role-playing, comment comparison, travel recommendations, etc. In addition, ChatGLM has developed a super large model in internal testing, with a parameter scale of 130 billion, which is one of the larger scales among current open-source dialogue models.The open-source nature of ChatGLM provides researchers and developers with a powerful tool for various dialogue scenarios and applications. Its performance and scale make it an ideal choice for handling complex dialogue tasks and generating high-quality dialogues. With the continuous development and improvement of ChatGLM, we can look forward to more innovative applications in the field of dialogue systems.Detailed IntroductionChatGLM-6B is an open-source, bilingual Chinese-English dialogue language model based on the

 

General Language Model (GLM)

architecture with 6.2 billion parameters. Combined with model quantization technology, users can perform local deployment on consumer-grade GPUs (the minimum required GPU memory under INT4 quantization is only 6GB). ChatGLM-6B uses similar technology to ChatGPT and optimizes for Chinese question-answering and dialogue. After training with about 1T bilingual Chinese-English tokens, supplemented by supervised fine-tuning, feedback self-learning, and human feedback reinforcement learning, the 6.2 billion parameter ChatGLM-6B can generate answers that are quite in line with human preferences. More information can be found in our blog.

To facilitate downstream developers to customize models for their own application scenarios, we have also implemented an efficient parameter fine-tuning method based on

P-Tuning v2 (User Guide) (using guide)

(User Guide) (using guide) (User Guide) (using guide) (User Guide)

If you want ChatGLM-6B to better fit your application scenario, welcome to participate in Badcase Feedback Program.

The open-source model of ChatGLM-6B aims to promote the development of large model technology together with the open-source community. We kindly request developers and everyone to comply with the open-source agreement and not use the open-source model, code, and derivatives based on open-source projects for any purposes that may harm the country and society, or for any services that have not been safety assessed and registered.Currently, the project team has not developed any applications based on ChatGLM-6B, including web, Android, Apple iOS, and Windows App applications.

Although the model has strived to ensure the legality and accuracy of the data at all stages of training, due to the relatively small scale of the ChatGLM-6B model and the influence of probabilistic randomness, it cannot guarantee the accuracy of the output content, and the model is prone to misdirection (see limitations).The project does not assume any risks and responsibilities arising from data security, public opinion risks, or any model misdirection, abuse, dissemination, or improper use caused by the open-source model and code.

Usage Methods

Hardware Requirements

Quantization Level Minimum GPU Memory(Inference) Minimum GPU Memory(Efficient Parameter Fine-tuning)
FP16 (Without Quantization) 13 GB 14 GB
INT8 8 GB 9 GB
INT4 6 GB 7 GB

Environment Installation

Install dependencies using pip:pip install -r requirements.txt, where transformers library version is recommended to be 4.27.1, but theoretically not lower than 4.23.1 is enough.

In addition, if you need to run the quantized model on CPU, you need to install gcc and openmp. Most Linux distributions are installed by default. For Windows, you can check TDM-GCC when installing openmp. Windows test environment gcc version is TDM-GCC 10.3.0, Linux is gcc 11.3.0. On MacOS, please refer to Q1.

Code Calls

You can call the ChatGLM-6B model to generate dialogue through the following code:

>>> from transformers import AutoTokenizer,AutoModel