为应对云层动态变化复杂、光伏功率与环境因素关系高度非线性的建模难题,提出一种融合 Transformer 与 Mamba 的多模态深度学习框架,通过联合建模天空图像的时空特征与红外光谱特征,实现更高精度的光伏功率预测。该研究采用Transformer建模云层运动的时序动态,利用Mamba提取红外光谱信息,并通过跨模态特征融合建立天空图像与发电功率之间的非线性关联。实验基于斯坦福天空图像与光伏发电数据集,验证了该模型在天空图像预测和光伏发电预测两方面的性能。消融实验结果表明,结合红外光谱、时序建模和多尺度特征提取的完整模型在图像预测任务中取得最优均方根误差（13.04 kW）和结构衡量指标（0.94653）,同时在光伏功率预测任务中相较现有方法误差降低4.5%。在典型多云天气场景下,模型能较好地跟踪云层遮挡引起的光伏功率波动趋势。
Abstract
Accurately forecasting sky imagery and its impact on solar power generation is essential for intelligent grid scheduling and the efficient management of renewable energy resources. However, capturing the dynamic evolution of cloud patterns and their complex, nonlinear relationship with photovoltaic （PV） power output remains a formidable challenge. To tackle this problem, we propose a multimodal deep learning framework that integrates Transformer and Mamba architectures, effectively leveraging both spatiotemporal features from sky images and infrared spectral information to improve forecasting precision. Within this framework, the Transformer is responsible for modeling the temporal dynamics of cloud motion, while Mamba focuses on extracting salient features from infrared spectra. A cross-modal feature fusion strategy is employed to learn the intricate nonlinear mapping between sky imagery and power output. We evaluate our model on the Stanford sky image and PV generation datasets, conducting extensive experiments that demonstrate superior performance in both sky image forecasting and PV power prediction tasks. Ablation studies reveal that the full model—incorporating infrared data, temporal modeling, and multi-scale feature extraction—achieves an RMSE of 13.04kW and an SSIM of 0.94653 for image prediction. Moreover,the proposed method achieves an MAE of 1.50 kW, approximately 4.5% lower than that of the second-best Tau model.Under typical cloudy weather conditions, the model is able to effectively track the trend of photovoltaic power fluctuations caused by cloud occlusion.
关键词
光伏发电 /
图像解读 /
多模态特征融合 /
状态空间方法
Key words
PV power generation /
photointerpretation /
multimodal feature fusions /
state space methods
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 周勃, 李二超. 一种考虑风光不确定性的热电混合共享储能双层优化配置方法[J]. 太阳能学报, 2025, 46(3): 189-198.
Zhou B, Li E C.A two-layer optimal allocation method for hybrid shared energy storage considering the uncertainty of wind power and photovoltaic[J]. Acta Energiae Solaris Sinica, 2025, 46(3): 189-198.
[2] Zhong Y J, Wu Y K.Short-term solar power forecasts considering various weather variables[C]//2020 International Symposium on Computer, Consumer and Control (IS3C). Taichung City, Taiwan, China, 2020: 432-435.
[3] 刘兴冉, 张宏晔, 闫海明, 等. 区域复杂地形晴日太阳辐射估算研究[J]. 太阳能学报, 2022, 43(8): 174-180. Liu X R, Zhang H Y, Yan H M, et al. Estimation of regional solar radiation on clear days over complex terrain[J]. Acta Energiae Solaris Sinica, 2022, 43(8): 174-180.
[4] 徐姝影, 张哲, 袁佳伟, 等. 考虑光伏跟踪装置的混合可再生能源系统优化与评价[J]. 太阳能学报, 2025, 46(3): 123-131. Xu S Y, Zhang Z, Yuan J W, et al. Optimization and evaluation of hybrid renewable energy systems considering PV tracking devices[J]. Acta Energiae Solaris Sinica, 2025, 46(3): 123-131.
[5] Vaswani A, Shazeer N, Parmar N, et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
[6] Gu A, Dao T, Ermon S, et al.HiPPO: recurrent memory with optimal polynomial projections[PP/OL]. V2. arXiv (2020-10-23)[2026-06-09]. https://doi.org/10.48550/arXiv.2008.07669.
[7] Jing T, Chen S L, Navarro-Alarcon D, et al.SolarFusionNet: enhanced solar irradiance forecasting via automated multi-modal feature selection and cross-modal fusion[J]. IEEE Transactions on Sustainable Energy, 2025, 16(2): 761-773.
[8] Wei X S, Yue D, Hancke G P, et al.Ultra short-term solar irradiance forecast based on multimodal data fusion and fuzzification[J]. IEEE Transactions on Industrial Informatics, 2025, 21(4): 3256-3265.
[9] Zhao X H, Fan H Y, Guo S, et al.Sky image-informed hybrid model for ultra-short-term prediction of photovoltaic power output[C]//4th Energy Conversion and Economics Annual Forum (ECE Forum 2024). Beijing, China, 2025: 375-379.
[10] Meddahi A, Tuomiranta A, Guillon S.Skill-driven data sampling and deep learning framework for minute-scale solar forecasting with sky images[J]. Solar RRL, 2025, 9(4): 2400664.
[11] 王东风, 李青博, 张博洋, 等. 考虑季节性与趋势特征的光伏功率预测模型研究[J]. 太阳能学报, 2025, 46(3): 348-356. Wang D F, Li Q B, Zhang B Y, et al. Research on photovoltaic power prediction model considering seasonality and trend characteristics[J]. Acta Energiae Solaris Sinica, 2025, 46(3): 348-356.
[12] Li H, Ma G, Wang B, et al.Multi-modal feature fusion model based on TimesNet and T2T-ViT for ultra-short-term solar irradiance forecasting[J]. Renewable Energy, 2025, 240: 122192.
[13] Hategan S M, Stefu N, Petreus D, et al.Short-term forecasting of PV power based on aggregated machine learning and sky imagery approaches[J]. Energy, 2025, 316: 134595.
[14] 黄政之,厉小润,张天文. 基于可见光与红外图像的光伏阵列组串热斑检测方法[J]. 太阳能学报, 2025, 46(3): 454-460.
Huang Z Z, Li X R, Zhang T W.Group hot spot detection of photovoltaic array based on visible and infrared thermal images[J]. Acta Energiae Solaris Sinica, 2025, 46(3): 454-460.
[15] 李宏扬, 高丙朋. 基于改进VMD和SNS-Attention-GRU的短期光伏发电功率预测[J]. 太阳能学报, 2023, 44(8): 292-300.
Li H Y, Gao B P.Short-term PV power forecasting based on improved VMD and SNS-Attention-GRU[J]. Acta Energiae Solaris Sinica, 2023, 44(8): 292-300.
[16] Nie Y H, Li X T, Scott A, et al.SKIPP’D: a SKy Images and Photovoltaic Power Generation Dataset for short-term solar forecasting[J]. Solar Energy, 2023, 255: 171-179.
[17] Tan C, Gao Z Y, Wu L R, et al.Temporal attention unit: towards efficient spatiotemporal predictive learning[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, BC, Canada, 2023: 18770-18782.
[18] Rao Y M, Zhao W L, Tang Y S, et al. HorNet: efficient high-order spatial interactions with recursive gated convolutions[PP/OL]. V3. arXiv (2022-10-11)[ 2026-06-09]. https://doi.org/10.48550/arXiv.2207.14284.
[19] Yu G Z, Lu L, Tang B, et al.Ultra-short-term wind power subsection forecasting method based on extreme weather[J]. IEEE Transactions on Power Systems, 2023, 38(6): 5045-5056.
[20] Yao T C, Wang J, Wu H Y, et al.A photovoltaic power output dataset: Multi-source photovoltaic power output dataset with Python toolkit[J]. Solar Energy, 2021, 230: 122-130.
[21] Li Y Y, Song L D, Zhang S, et al.A TCN-based hybrid forecasting framework for hours-ahead utility-scale PV forecasting[J]. IEEE Transactions on Smart Grid, 2023, 14(5): 4073-4085.
[22] LI Q Q, XU Y, CHEW B S H, et al. An integrated missing-data tolerant model for probabilistic PV power generation forecasting[J]. IEEE Transactions on Power Systems, 2022, 37(6): 4447-4459.
基金
国家自然科学基金（62461052）; 青海省“揭榜挂帅”重大科技专项（2024-GX-A3)
PDF(1642 KB)
