OpenBMB Official Website, Open Source Toolchain for Large Models, Build a Library of Large-Scale Pretrained Language Models and Related Tools
What is OpenBMB?
OpenBMB, short for Open Lab for Big Model Base, aims to build a library of large-scale pretrained language models and related tools, accelerate the training, fine-tuning, and inference of models over 100 billion parameters, and lower the entry barrier for large models.
OpenBMB Official Website:https://www.openbmb.org
OpenBMB GitHub Project Address:https://github.com/OpenBMB
OpenBMB, short for Open Lab for Big Model Base, aims to build a library of large-scale pretrained language models and related tools, accelerate the training, fine-tuning, and inference of models over 100 billion parameters, lower the entry barrier for large models, work together with developers worldwide to form an open-source community for large models, promote the development of the large model ecosystem, achieve the standardization, popularization, and practical application of large models, and bring large models into thousands of households.
OpenBMB will strive to build an open-source community for large models, unite a large number of developers to continuously improve the full-process supporting tools for large models from training, fine-tuning, inference to application. Based on the early work of the contributor team, OpenBMB has designed a full-process R&D framework for large models and initially developed related tools, which perform their respective functions, collaborate with each other, and collectively achieve efficient computation in the full process of training, fine-tuning, and inference of large models.
OpenBMBFull NameOpen Lab for Big Model Base, aims tobuild a library of large-scale pretrained language models and related tools, accelerate the training, fine-tuning, and inference of models over 100 billion parameters, lower the entry barrier for large models, work together with developers worldwide to form an open-source community for large models, promote the development of the large model ecosystem, achieve thestandardization,popularizationandpractical application,bringing large models into thousands of households.
BMTrain
The "engine" for large model training. BMTrain performs efficient pretraining and fine-tuning of large models. Compared to frameworks like DeepSpeed, BMTrain can save up to 90% of the training model cost.
BMCook
Large model "weight loss" toolkit. BMCook performs efficient compression of large models, improving runtime efficiency. Through a combination of quantization, pruning, distillation, and expertization algorithms, it can maintain over 90% of the original model's effectiveness while accelerating model inference by 10 times.
BMInf
Large model inference with a thousand-dollar graphics card. BMInf achieves low-cost and efficient inference computation for large models, allowing for the inference of models with over 100 billion parameters using a single thousand-dollar graphics card (GTX 1060).
OpenPrompt
A powerful tool for large model prompt learning. OpenPrompt provides a unified interface prompt learning template language, its composable and modular nature allows you to easily deploy prompt learning methods to drive large models.
OpenDelta
"Small" parameters to move "large" models. OpenDelta performs efficient fine-tuning of large models, achieving full parameter fine-tuning effects with only a small update of parameters (less than 5%).
ModelCenter
Large model repository. ModelCenter, based on the BMTrain tool, has implemented a series of pretrained language models, supporting efficient, low-cost, and highly scalable model fine-tuning and distributed training.
OpenBMB Capability System
Plan and act, OpenBMB will build adata, tools, models, protocolsfour levels to buildapplication convenience, comprehensive capabilities, and standardized uselarge-scale pretrained model library.OpenBMBCapability SystemSpecifically includes:Build large-scale dataAutomatic collection, automatic cleaning, and efficient storagemodules and related tools, providing data support for large model training.Focus onmodel training, model fine-tuning, model inference, and model applicationfour major scenarios of large models, launch supporting open-source toolkit, improve efficiency at each stage, and reduce computational and labor costs.Build an open-source large model library supported by the OpenBMB tool, including general large models such as BERT, GPT, T5, and open-source large models such as CPM, EVA, GLM, and continuously add new models to form a comprehensive model capability.▶Protocol LayerRelease General model license agreement, regulate and protect the rights and obligations of publishers and users in the process of publishing and using large models, and the initial draft of the agreement has been open-sourced (https://www.openbmb.org/license).Large model-related tools play a core role in the OpenBMB capability system. OpenBMB will strive to build alarge model open-source community, unite a large number of developers to continuously improve the full-process supporting tools for large models from training, fine-tuning, inference to application. Based on the early work of the initiating team, OpenBMB has designed a full-process R&D framework for large models and initially developed related tools, which perform their respective functions, collaborate with each other, and collectively achieve efficient computation in the full process of training, fine-tuning, and inference of large models.OpenBMB Tools▶
Model Training Suite
BMData: Large model "raw material" collectorBMData performshigh-quality data cleaning, processing, and storage, providing comprehensive and integrated data support for large model training.BMTrain:Large model training "engine"BMTrain performsefficient pretraining and fine-tuning of large models. Compared to frameworks like DeepSpeed, BMTrain can save up to 90% of the training model cost.BMCook:Large model "weight loss" toolkitBMCook performsefficient compression of large models, improving runtime efficiency. Through a combination of quantization, pruning, distillation, and expertization algorithms, it can maintain over 90% of the original model's effectiveness while accelerating model inference by 10 times.BMCook:Large Model "Slimming" ToolkitBMCook performsEfficient compression of large modelsto enhance runtime efficiency. Through quantization, pruning, distillation, expert