sklearn

Official Scikit-learn Website, a free software machine learning library for Python programming language

Sklearn Official Website:https://scikit-learn.org/stable/index.html

Sklearn Chinese Guide Community:https://scikit-learn.org.cn/

Introduction to Common Sklearn Features

2.1 Clustering

Clustering is a typical unsupervised learning task but is also a common requirement in practical applications. Without providing the true labels of samples, clustering is performed based on certain features. According to the principles of clustering, it mainly includes: K-MEANS, Affinity Propagation, MeanShift, Spectral Clustering, Agglomerative Clustering, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Balanced Iterative Reducing and Clustering using Hierarchies (Birch), Gaussian Mixture Models (GMM), and Spectral Biclustering.

2.2 Dimensionality Reduction

Dimensionality reduction also belongs to unsupervised learning, which compresses existing features and the reduced features are not any of the original features in the feature matrix but are combined in some way. It aims to reduce dimensions while retaining as much original information as possible, usually using PCA, and the reduced features have no direct relationship with the original features, making model training no longer interpretable. According to the principles of dimensionality reduction, it mainly includes: standard PCA, Incremental PCA (IPCA), PCA using randomized SVD, Kernel PCA, Sparse PCA and MiniBatchSparsePCA, Non-negative Matrix Factorization, Independent Component Analysis (ICA), and Latent Dirichlet Allocation (LDA).

2.3 Metric Measures

Different metric measures can learn different optimal models. For different types of tasks, sklearn provides various metric measures, including:
· Classification tasks: Accuracy, the proportion of correctly classified samples among all samples; Precision and recall, two conflicting indicators, suitable for situations where the number of classification samples is unbalanced. To ensure a certain goal, only one of the two indicators can be selected; Harmonic mean F1, which takes both precision and recall into account;
· Regression tasks: Commonly used include MSE, MAE, F1-score;
· Clustering tasks: Silhouette Coefficient (no prior labels required, measured by the ratio of within-cluster distance to the nearest distance outside the cluster), Adjusted Rand Index (based on the true cluster labels and cluster labels).

2.4 Ensemble Learning Models

When the performance of basic learning models is difficult to meet the requirements, ensemble learning comes into play. Ensemble learning, as the name implies, is to integrate the results of multiple base learners to produce the final result. According to the process of integration, ensemble learning mainly includes three schools of thought:
· Bagging, also known as bootstrap aggregating, achieves parallel training of multiple differentiated base learners through bootstrap sampling (with replacement). Although each learner may not be outstanding, the final result obtained through voting can steadily improve performance. When the base learner takes decision trees, the ensemble learning model based on bagging is Random Forest. Another way corresponding to bagging is without replacement sampling, which is called pasting, but it is less commonly used;
· Boosting, also known as boosting method. Base learners are combined in series, trying to reduce the bias of the combined learning model. Several weak learners are combined into a powerful ensemble model. (eg: adaboost, Gradient Tree Boosting) Unlike bagging, which trains multiple base learners in parallel and independently, boosting is based on the training results of the previous step to train a better model, which belongs to a serial mode. According to different implementation details, there are two boosting models, namely Adaboost and GBDT. The core difference between the two lies in the focus of boosting: the former focuses on previously misclassified samples, while the latter focuses on previously unlearned residuals. Another popular XGBoost is an improvement on GBDT, with the same core idea;
· Stacking, also known as stacking method, is similar to bagging but different in some aspects: stacking also trains multiple base learners in parallel and independently, and then uses these training results as features for further learning. It is somewhat similar to multi-layer neural networks in deep learning.

2.5 Sample Datasets

Sklearn provides some classic datasets, mainly focusing on two classic tasks: classification and regression. Through these datasets, machine learning tasks can be quickly set up and model performance can be compared.
Brief introduction to common datasets:
· load_breast_cancer: Breast cancer dataset, a binary classification task with labels 0 or 1;
· load_iris: Classic iris dataset, with continuous numerical variables as features and labels 0/1/2 as a three-classification task;
· load_wine: Wine dataset, with slightly unbalanced sample numbers;
· load_digits: Small handwritten digit dataset (it is called small because there is also a large handwritten digit dataset mnist), containing 0-9 labels, with balanced samples;
· load_boston: Boston housing price dataset, suitable for regression tasks.

2.6 Data Preprocessing

Each model in sklearn has a standardized data input/output format, generally using np.array and pd.dataframe as the standard format, so some string discrete labels cannot be used directly for model training; at the same time, in order to accelerate the speed of model training and ensure training accuracy, data preprocessing is often required, and common data preprocessing functions include:
· MinMaxScaler: Min-Max normalization;
· StandardScaler: Standardization normalization, mainly suitable for situations where there may be extreme values, at which time using MinMaxScaler may cause other values to be transformed too concentrated due to a single outlier, while using standard normal distribution to dimensionless can effectively avoid this problem;
· Binarizer: Binarization processing, suitable for discretizing continuous variables;
· OneHotEncoder: One-hot encoding, a classic encoding method, suitable for discrete labels without clear size relative relationships;
· Ordinal: Numerical encoding, suitable for certain label encoding as numbers that do not affect model understanding and training.

2.7 Feature Selection

In machine learning, there is a classic saying: "Data and features determine the upper limit of learning, and models and algorithms only approximate this upper limit." This shows the importance of feature engineering in machine learning. Several common feature selection methods include: from_model, VarianceThreshold, SelectKBest.

2.8 Model Selection

Model selection is an important step in machine learning, involving operations such as dataset splitting, parameter adjustment, and validation. Corresponding common functions include:
· train_test_split: One of the common operations, splitting the dataset and test set, which can set the split ratio;
· cross_val_score: Cross-validation, with a default of 5-fold, which is equivalent to dividing the dataset into 5 equal parts, and using one of them as the test set, the rest as the training set for training and scoring, and finally returning K scores;
· GridSearchCV: A common method for parameter tuning, using a dictionary type to set a set of candidate parameters, and specifying a measurement standard, and finally returning the parameters with the highest score.

2.9 Basic Learning Models

Classification and regression tasks are classic scenarios in machine learning, which belong to supervised learning. Classic learning algorithms mainly include 5 types:
· Linear models, corresponding to linear regression in regression tasks, and logistic regression or log-odds regression in classification tasks, which essentially fit the log-odds through linear regression to achieve binary classification;
· K-Nearest Neighbors, the simplest and most intuitive machine learning model, which does not require training (lazy model), and simply judges the analogy or fitting result by the samples around its own position;
· Support Vector Machines, a classic machine learning model, which originated from linear classification and achieves the most reliable classification boundary by maximizing the margin. It is said in the industry that support vector machines have three treasures: margin, dual kernel function. The "margin" upgrades the hard margin to the soft margin to solve the linearly inseparable scenario with outliers, "dual" is a small trick in solving the Lagrange problem in the optimization process, and the kernel function is the core of support vector machines. Through the kernel function, it can be upgraded from linearly separable to linearly inseparable, while avoiding dimension disaster;
· Naive Bayes, derived from Bayes' theorem in probability theory, the model training process is to fit the probability distribution of each feature, and the prediction process is to label the analogy with the highest probability, which is a pure probability-based classification model. While models like logistic regression, K-Nearest Neighbors, Support Vector Machines, and Decision Trees can also predict the probability of various categories, they are not purely probabilistic;
· Decision Trees, an intuitive and powerful machine learning model, the training process mainly includes feature selection, splitting, and pruning. Typical decision trees are ID3, C4.5, and CART, among which CART trees can be used for both classification and regression. More importantly, decision trees not only have research value in themselves but are also the base learners of many ensemble learning models.

Installation Guide

There are several ways to install scikit-learn:

• Install the latest official version. This is the best method for most users. It will provide a more stable version, and the precompiled software package is suitable for most platforms.
• Install the scikit-learn version provided by the computer operating system or Python distribution. This is a quick choice for users whose computer operating system or Python distribution is compatible with scikit-learn. It may not provide the latest release version.
• Build the software package from source code. This is the best choice for users who want the latest and most powerful features and are not afraid to run new code. This is also the best choice for users who want to contribute to the project.

Install the latest version

Operating System: Windows

Package Manager: pip

For example, install the 64-bit version of Python 3 from https://www.python.org.

Then run:

1. pip install -U scikit-learn


You can use the following statement to check

1. Python