Conjugational Stability and Symmetry-Invariant Transformations in Artificial Neural Networks
DOI:
https://doi.org/10.70862/CSIR.2026.0301-18Keywords:
algebraic stabilization, linear transformations, conjugational equivalence, symmetry-invariant networks, transformation robustness, symmetries and equivarianceAbstract
This paper examines the properties of algebraically stabilized linear transformations in artificial neural networks (ANNs) under a change of basis and their compatibility with symmetry-invariant architectures. We investigate the conditions under which a stabilizing transformation of the form W^k=I+μD, defined over integer matrices, preserves its properties under conjugation, i.e., when passing to an equivalent form W'=B^(-1) WB in a new basis. It is shown that such transformations form a closed class with respect to conjugational equivalence and can be employed in environments with diverse geometric and symmetry requirements. Additionally, we analyze the role of symmetric actions on the input space and define conditions for the invariance of transformations with respect to specific groups of transformations. Both formal algebraic properties and the geometric and semantic interpretation of stability in this context are considered. Different types of stability are presented – algebraic, conjugational, and symmetry-based – along with the relationships between them. The results are relevant to the design of ANN architectures with predictable behavior, robustness under transformations, and compatibility with predefined symmetric structures.
References
Gogianu, F.; Berariu, T.; Rosca, M.C.; Clopath, C.; Busoniu, L.; Pascanu, R. Spectral normalisation for deep reinforcement learning: An optimisation perspective. In Proceedings of the International Conference on Machine Learning, 2021; pp. 3734–3744. Retrieved January 8, 2026, from https://proceedings.mlr.press/v139/gogianu21a/gogianu21a.pdf
Keriven, N. Not too little, not too much: A theoretical analysis of graph (over)smoothing. Advances in Neural Information Processing Systems 2022, 35, 2268–2281. Retrieved January 8, 2026, from https://proceedings.neurips.cc/paper_files/paper/2022/file/0f956ca6f667c62e0f71511773c86a59-Paper-Conference.pdf
Shi, Z.; Mettes, P.; Maji, S.; Snoek, C.G. On measuring and controlling the spectral bias of the deep image prior. International Journal of Computer Vision 2022, 130, 885–908. https://doi.org/10.1007/s11263-021-01572-7
Zhai, S.; Likhomanenko, T.; Littwin, E.; Busbridge, D.; Ramapuram, J.; Zhang, Y.; Susskind, J.M. Stabilizing transformer training by preventing attention entropy collapse. In Proceedings of the International Conference on Machine Learning, July 2023; pp. 40770–40803. Retrieved January 8, 2026, from https://proceedings.mlr.press/v202/zhai23a/zhai23a.pdf
Purnomo, A.; Tjandrasa, H. Improved deep learning architecture with batch normalization for EEG signal processing. JUTI: Jurnal Ilmiah Teknologi Informasi 2021, 19, 19–27. https://doi.org/10.12962/j24068535.v19i1.a1023
Ziaee, A.; Çano, E. Batch layer normalization: A new normalization layer for CNNs and RNNs. In Proceedings of the 6th International Conference on Advances in Artificial Intelligence, October 2022; pp. 40–49. https://doi.org/10.1145/3571560.3571566
Peng, H.; Yu, Y.; Yu, S. Re-thinking the effectiveness of batch normalization and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence 2023, 46, 465–478. https://doi.org/10.1109/TPAMI.2023.3319005
Ruhland, J.B.; Masoudian, I.; Heider, D. Enhancing deep neural network training through learnable adaptive normalization. Knowledge-Based Systems 2025, 113968. https://doi.org/10.1016/j.knosys.2025.113968
Wang, R.; An, S.; Liu, W.; Li, L. Invertible residual blocks in deep learning networks. IEEE Transactions on Neural Networks and Learning Systems 2023, 35, 10167–10173. https://doi.org/10.1109/TNNLS.2023.3238397
Selvaraju, K.; Rajamani, S. New approach to underwater image enhancement using modified residual blocks in generator architecture for improved cycle generative adversarial networks. Proceedings of the Bulgarian Academy of Sciences 2024, 77, 73–81. https://doi.org/10.7546/CRABS.2024.01.09
Eskandari, H.; Imani, M.; Moghaddam, M.P. A deep residual network integrating entropy-based wavelet packet ensemble model for short-term electrical load forecasting. Energy 2025, 314, 134168. https://doi.org/10.1016/j.energy.2024.134168
Yotov, K., Hadzhikolev, E., & Hadzhikoleva, S. (2026). Algebraic Stabilization of Linear Transformations in Artificial Neural Networks. Mathematics, 14(4), 623. https://doi.org/10.3390/math14040623
Huang, L., Yang, D., Lang, B., & Deng, J. (2018). Decorrelated batch normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 791-800). https://doi.org/10.1109/CVPR.2018.00089
Mobasseri, B. G., & Lulu, A. (2021). Radiometric identification of signals by matched whitening transform. Sensors, 21(24), 8398. https://doi.org/10.3390/s21248398
Zhang, C., Zheng, Y., Guo, B., Li, C., & Liao, N. (2021). SCN: a novel shape classification algorithm based on convolutional neural network. Symmetry, 13(3), 499. https://doi.org/10.3390/sym13030499
Zhang, S., Nezhadarya, E., Fashandi, H., Liu, J., Graham, D., & Shah, M. (2021). Stochastic whitening batch normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10978-10987). https://doi.org/10.1109/CVPR46437.2021.01083
Xu, J., Kim, H., Rainforth, T., & Teh, Y. (2021). Group equivariant subsampling. Advances in Neural Information Processing Systems, 34, 5934-5946. Retrieved January 8, 2026, from https://proceedings.neurips.cc/paper/2021/file/2ea6241cf767c279cf1e80a790df1885-Paper.pdf
Favoni, M., Ipp, A., Müller, D. I., & Schuh, D. (2022). Lattice gauge equivariant convolutional neural networks. Physical Review Letters, 128(3), 032003. https://doi.org/10.1103/PhysRevLett.128.032003
Mirkes, E. M., Bac, J., Fouché, A., Stasenko, S. V., Zinovyev, A., & Gorban, A. N. (2022). Domain adaptation principal component analysis: base linear method for learning with out-of-distribution data. Entropy, 25(1), 33. https://doi.org/10.3390/e25010033
Shabbir, A., Ali, N., Ahmed, J., Zafar, B., Rasheed, A., Sajid, M., ... & Dar, S. H. (2021). Satellite and scene image classification based on transfer learning and fine tuning of ResNet50. Mathematical Problems in Engineering, 2021(1), 5843816. https://doi.org/10.1155/2021/5843816
Chakraborty, S., Mondal, R., Singh, P. K., Sarkar, R., & Bhattacharjee, D. (2021). Transfer learning with fine tuning for human action recognition from still images. Multimedia Tools and Applications, 80(13), 20547-20578. https://doi.org/10.1007/s11042-021-10753-y
Araújo, R. L., Araújo, F. H. D., & Silva, R. R. E. (2022). Automatic segmentation of melanoma skin cancer using transfer learning and fine-tuning. Multimedia Systems, 28(4), 1239-1250. https://doi.org/10.1007/s00530-021-00840-3
Prottasha, N. J., Sami, A. A., Kowsher, M., Murad, S. A., Bairagi, A. K., Masud, M., & Baz, M. (2022). Transfer learning for sentiment analysis using BERT based supervised fine-tuning. Sensors, 22(11), 4157. https://doi.org/10.3390/s22114157
Bökman, G., Kahl, F., & Flinth, A. (2022). Zz-net: A universal rotation equivariant architecture for 2d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10976-10985). https://doi.org/10.1109/CVPR52688.2022.01070
Sanborn, S., & Miolane, N. (2023). A general framework for robust g-invariance in g-equivariant networks. Advances in Neural Information Processing Systems, 36, 67103-67124. Retrieved January 8, 2026, from https://proceedings.neurips.cc/paper_files/paper/2023/file/d42523d621194ba54dda098669645f91-Paper-Conference.pdf
Navon, A., Shamsian, A., Achituve, I., Fetaya, E., Chechik, G., & Maron, H. (2023, July). Equivariant architectures for learning in deep weight spaces. In International Conference on Machine Learning (pp. 25790-25816). PMLR. Retrieved January 8, 2026, from https://proceedings.mlr.press/v202/navon23a/navon23a.pdf
Kim, J., Nguyen, D., Suleymanzade, A., An, H., & Hong, S. (2023). Learning probabilistic symmetrization for architecture agnostic equivariance. Advances in Neural Information Processing Systems, 36, 18582-18612. Retrieved January 8, 2026, from https://proceedings.neurips.cc/paper_files/paper/2023/file/3b5c7c9c5c7bd77eb73d0baec7a07165-Paper-Conference.pdf
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014). Speeding up Convolutional Neural Networks with Low Rank Expansions. British Machine Vision Conference. BMVA Press. http://dx.doi.org/10.5244/C.28.88
Deng, L., Li, G., Han, S., Shi, L., & Xie, Y. (2020). Model compression and hardware acceleration for neural networks: A comprehensive survey. Proceedings of the IEEE, 108(4), 485-532. https://doi.org/10.1109/JPROC.2020.2976475
Wu, R., Guo, X., Du, J., & Li, J. (2021). Accelerating neural network inference on FPGA-based platforms—A survey. Electronics, 10(9), 1025. https://doi.org/10.3390/electronics10091025
Sui, X., Lv, Q., Bai, Y., Zhu, B., Zhi, L., Yang, Y., & Tan, Z. (2022). A hardware-friendly low-bit power-of-two quantization method for CNNs and its FPGA implementation. Sensors, 22(17), 6618. https://doi.org/10.3390/s22176618
Wang, C., & Luo, Z. (2022). A review of the optimal design of neural networks based on FPGA. Applied Sciences, 12(21), 10771. https://doi.org/10.3390/app122110771
Tasci, M., Istanbullu, A., Tumen, V., & Kosunalp, S. (2025). FPGA-QNN: quantized neural network hardware acceleration on FPGAs. Applied Sciences, 15(2), 688. https://doi.org/10.3390/app15020688
Wang, Y., & Zhao, H. (2025). An Improved Strategy for Data Layout in Convolution Operations on FPGA-Based Multi-Memory Accelerators. Electronics, 14(11), 2127. https://doi.org/10.3390/electronics14112127
Liao, S., Samiee, A., Deng, C., Bai, Y., & Yuan, B. (2019, May). Compressing deep neural networks using toeplitz matrix: Algorithm design and fpga implementation. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1443-1447). IEEE. https://doi.org/10.1109/ICASSP.2019.8683556
Wang, E., Davis, J. J., Zhao, R., Ng, H. C., Niu, X., Luk, W., ... & Constantinides, G. A. (2019). Deep neural network approximation for custom hardware: Where we've been, where we're going. ACM Computing Surveys (CSUR), 52(2), 1-39. https://doi.org/10.1145/3309551
Zacchigna, F. G., Lew, S., & Lutenberg, A. (2024). Flexible quantization for efficient convolutional neural networks. Electronics, 13(10), 1923. https://doi.org/10.3390/electronics13101923
Zhang, L., Krestinskaya, O., Fouda, M. E., Eltawil, A. M., & Salama, K. N. (2025). Quantized convolutional neural networks: a hardware perspective. Frontiers in Electronics, 6, 1469802. https://doi.org/10.3389/felec.2025.1469802
Mondal, S., & Sivakumar, K. C. (2025). A Survey of Z-Matrices and New Results on the Subclass of F0-Matrices. In Inverse Problems, Regularization Methods and Related Topics: A Volume in Honour of Thamban Nair (pp. 303-327). Singapore: Springer Nature Singapore. https://doi.org/10.1007/978-981-97-7989-5_14
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2026 Kostadin Yotov, Emil Hadzhikolev (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
