Face Photo and Sketch Synthesis Using CycleGAN: The Role of Residual Blocks and Spectral Normalization

Abstract

This study investigates the problem of face photo and sketch synthesis using CycleGAN, with a particular focus on architectural choices that affect model performance. A major challenge in this field is the limited availability of paired face photo-sketch datasets, which makes it difficult to develop and evaluate effective models. To address this limitation, this research demonstrates how to generate face sketches and photos from small datasets, offering a practical solution for expanding existing data collections in future applications, particularly in law enforcement and forensic sketch analysis. Four CycleGAN configurations were designed by varying the number of residual blocks in the generator and applying spectral normalization to the discriminator. Experiments were conducted on the CUHK Face Sketch Database, with quantitative evaluations using SSIM and PSNR, as well as qualitative assessments via human visual inspection. The results show that increasing generator depth and incorporating spectral normalization in the discriminator both improve image quality and training stability, although these benefits come at the cost of increased training time. Notably, the configuration with ResNet9 and spectral normalization consistently achieved the highest scores, with SSIM and PSNR values of 0.702 and 18.53 for sketch-to-photo translation, and 0.702 and 17.35 for photo-to-sketch translation, and produced the most visually convincing results in both tasks. These findings highlight the importance of architectural design choices for effective face photo-sketch translation and provide guidance for future work in related applications.