Multi-label active learning (MLAL) is to use AL on multi-label learning (MLL) problems. Each instance has multiple classification labels. We note that MLAL could be seen as a degeneration from multi-task active learning (MTAL) when all the tasks are classification. Sometimes these two fields are focus on the same MLAL problems but sometimes are not. In this chapter, we only discuss the MLAL problems in spite of what terminology the papers use. Different from the general MTAL, MLAL usually utilize the relationship between labels.
We can divide the MLAL works into three types by the query type:
- Query all the labels of the selected unlabeled instances.
- Query the specific label of the selected instance. More efficient and minimize the waste of budget.
- Others
| MLAL query types | Works |
|---|---|
| Instance query | MML/MMC/BMAL/Adaptive/CVIRS |
| Instance-label pair query | 2DAL/AUDI/QUIRE |
| Others | AURO |
- Multi-label svm active learning for image classification [2004, ICIP]: ML, MML. First MLAL work. The selected image set consists of the images the sum of whose expected loss values are largest. (133)
- Effective multi-label active learning for text classification [2009, SIGKDD]: MMC The framework use SVM to predict, and a LR was used to predict the number of the labels of the instance. Select the unlabeled data which can lead to the largest reduction of the expected model loss (version space of SVM). Expected loss for multi-label data is approximated by summing up losses on all labels according to the most confident result of label prediction. (150)
- Optimal batch selection for active learning in multi-label classification [2011, ACMMM] BMAL Batch-mode MLAL. Selected batch of points have minimal redundancy among them. They design an uncertainty vector and uncertainty matrix to evaluate the redundancy between unlabeled points. (18)
- Active learning with multi-label svm classification [2013, IJCAI]: Measure the informativeness of an instance by combining the label cardinality inconsistency and the separation margin with a tradeoff parameter The two strategies are integrated into an adaptive framework of multi-label active learning Adaptive (88)
- Multi-Label Deep Active Learning with Label Correlation [2018, ICIP]: Use LSTM to model the relations among labels. CNN is used to produce high level representation of the image and the LSTM models the label dependencies. Select k samples furnishing the maximum entropies to form batch B. (2)
- Effective active learning strategy for multi-label learning [2018, Neurocomputing]: The review part of this paper is quite good. Uncertainty Sampling based on Category Vector Inconsistency and Ranking of Scores (CVIRS). This strategy selects those unlabelled examples having the most unified uncertainty computed by means of the rank aggregation problem formulated, and at the same time, the most inconsistent predicted category vectors. (19)
- Granular Multilabel Batch Active Learning With Pairwise Label Correlation [2021, TSMC]: Granular batch mode-based ranking active model for the multilabel (GBRAML).
- A Gaussian Process-Bayesian Bernoulli Mixture Model for Multi-Label Active Learning [2021, NeurIPS]: GP-B2M
- Active learning in multi-label image classification with graph convolutional network embedding [2023, FGCS]
- Two-dimensional active learning for image classification [2008, CVPR]: 2DAL. Select sample-label pairs, rather than only samples, to minimize a multi-label Bayesian classification error bound. Propose a Kernelized Maximum Entropy Model (KMEM) to model label correlations. They showed that querying instance-label pairs is more effective. (143)
- Multi-Task Active Learning with Output Constraints [2010, AAAI]: Not like a conventional AL, it design a reward function to calculate VOI (value of information). Each possible labeling assignment for a task is first propagated to all relevant tasks reachable through constraints and the reward (VOI) is measured over all relevant reachable tasks. Generalize the entropy measure used in the classical single-task uncertainty sampling and also highlight the role of task inconsistency in multi-task active learning. (51)
- Active query driven by uncertainty and diversity for incremental multi-label learning [2013, ICDM]: AUDI. Propose an incremental MLL model by combining label ranking with threshold learning, and avoid retraining from scratch after every query. Propose to exploit both uncertainty and diversity in the instance space and label space with the incremental MLL model. In the instance space, they simultaneously evaluate the uncertainty with label cardinality inconsistency and the diversity with the number of labels not queried. In the labels space, the distance from a label to the thresholding dummy label is employed to evaluate the uncertainty. (42)
- Multi-label Image Classification via High-Order Label Correlation Driven Active Learning [2014, TIP]: HoAL. Introduce auxiliary compositional label to measure the score containing multi-correlations. The cross-label uncertainty on unlabeled data is defined based on KL divergence. Both single-label uncertainty and cross-label uncertainty are unified by the cross entropy measure. The informative example-label pair selection is formulated as a continuous optimization problem over selection variables with the consideration of label correlations. (39)
- Active learning by querying informative and representative examples [2014, TPAMI]: QUIRE. Introduce a label correlation matrix. Combining label correlation with the measures of representativeness and informativeness for query selection. (393)
- Cost-Effective Active Learning for Hierarchical Multi-Label Classification [2018, IJCAI]: A batch mode selection. A label hierarchy is pre-defined. The informativeness of instance-label pair is counted in the contribution of ancestor and dependent.
- Multi-label active learning based on submodular functions [2018, Neurocomputing]: Propose a query strategy by constructing a submodular function for the selected instance-label pairs, which can measure and combine the informativeness and representativeness
- Multiview Multi-Instance Multilabel Active Learning [2021, TNNLS]
- Cost-effective Batch-mode Multi-label Active Learning [2021, Neurocomputing]
- Multi-label active learning through serial-parallel neural networks [2022, KBS]
- Stable matching-based two-way selection in multi-label active learning with imbalanced data [2022, Information Sciences]
- Active Crowdsourcing for Multilabel Annotation [2022, TNNLS]
- A meta-framework for multi-label active learning based on deep reinforcement learning [2023, Neural Networks]
- BI-LAVA: Biocuration with Hierarchical Image Labeling through Active Learning and Visual Analysis [2023, Computer Graphics Forum]
- Active Batch Sampling for Multi-label Classification with Binary User Feedback [2023, WACV]
- Multi-Label Active Learning: Query Type Matters [2015, IJCAI]: AURO. AL strategies that select example-label pairs avoid information redundancy, but they may ignore the interaction between labels and can obtain a limited supervision from each query. They iteratively select one instance along with a pair of labels, and then query their relevance ordering, i.e., ask the oracle which of the two labels is more relevant to the instance.
- Active Refinement for Multi-Label Learning: A Pseudo-Label Approach [2021]
- Partial multi-label learning: exploration of binary ground-truth labels [2023, ICME]
In this case, the task is to predict the labels of bags of instances. The number of labels would be more than two.
- Active Learning in Incomplete Label Multiple Instance Multiple Label Learning [2021, Arxiv]
- Cost-effective multi-instance multilabel active learning [2021]