This project focused on verifying the findings and expanding upon the evaluation and training methods from the paper LICO: Explainable Models with Language-Image Consistency.
The main claims are that LICO:
-
- enhances interpretability by producing more explainable saliency maps in conjunction with a post-hoc explainability method and
-
- improves image classification performance without computational overhead during inference.
We reproduced the key experiments conducted by Lei et al. however, the obtained results do not support the original claims. Additionally, we identify a vulnerability in the paper’s main evaluation method that favors non-robust models, and propose robust experimental setups for quantitative analysis using three new metrics. Furthermore, we undertake additional studies on LICO’s training methodology to enhance its interpretability
Paper will be available soon (currently under review at TMLR).