一种融合多模态扩散生成与轻量化分割的光伏缺陷智能诊断框架

Abstract Ensuring the long-term reliability and efficiency of photovoltaic(PV) systems requires accurate and intelligent defect monitoring strategies. To address these issues, this study proposes an innovative defect image generation method called CAM-Diffuse, which combines binary mask constraints with text-vision-based multimodal feature fusion. This method enables the generation of high-fidelity and controllable defect images, effectively expanding the training dataset and enhancing the model’s generalization ability.Furthermore, this study introduces LightSegDETR, a lightweight instance segmentation network. The network integrates the DGBlock module, which combines DWConv and GhostConv to optimize computational efficiency. In the neck of the network, DynamicGhost (dynamic adaptive adjustment of ghost convolution) and AdaptiveWT (adaptive wavelet high and low-frequency feature fusion) techniques are used for feature fusion. In the head, the self-attention mechanism is combined with SEBAttention (a multi-scale dual-attention strategy) to achieve joint adaptive weighting.Compared to the Enhanced Baseline, LightSegDETR reduces parameters (Params) and memory (RAM) by 50%, and computational load (GFLOPs) by 34.4%, while achieving improvements in accuracy: mAP50detect, mAP50-95detect, mAP50seg, and mAP50-95seg increase by 1.1%, 1.3%, 1.0%, and 0.8%, respectively. LightSegDETR achieves 28 FPS on Jetson Nano with state-of-the-art accuracy-efficiency, enabling robust real-time edge deployment, demonstrating strong potential for edge deployment and cost-effective PV performance monitoring in practical applications.