Hinet is a ResNet based network which is specialized in image restoration, designed by Liangyu Chen et.al. The most important part of HINet is that it uses its own regularization method which normalizes only half of the instances. HINet is reaching SOTA in various denoising and image restoration tasks, and showed quite amazing performance on my own DACON project.
https://arxiv.org/abs/2105.06086
HINet: Half Instance Normalization Network for Image Restoration
In this paper, we explore the role of Instance Normalization in low-level vision tasks. Specifically, we present a novel block: Half Instance Normalization Block (HIN Block), to boost the performance of image restoration networks. Based on HIN Block, we de
arxiv.org
1. HINet
Normalization is usally included in deep learning models as batch normalization layers, to make training the model more stable. However, in super resolution tasks like denoising or image restoration, using batch normalization in those are not that useful in increasing the performance. To overcome this main obstacle, researchers have invented a new model which adopts advanced instance normalizing method called Half Instance Normalization. As this method is the core of HINet, lets take a deep look on half instance normalizing.
2. Half Instance Normalization
Half Instance Normalization is quite straightforward to understand its mechanism. As it is named, we not normalize all the instance channels, but only half of those channels (in Conv Layers - obviously because its an image task). The block which performs half instance normalization is called Half Instance Normalization Block(HIN Block) and HINet is built upon using various blocks including SAM, CSFF, Res Block and our HIN Block.
3. The Architecture
HINET
As we can see on the image of the full architecture above, input image is feeded to the model twice, each for one stage. Each stage is a encoder-decoder styled sub-network which includes HINBlock on encoder section and ResBlock on decoder section. The data transfered between those two stages is passed through Supervised Attention Module(SAM) which extracts the important information from the input image. Second layer receives both raw input and core information extracted by stage1 and produces the final prediction as a residual to original image.
HIN Block
As mentioned above, batch normalization is not useful in tasks which is sensetive to small variances. To catch both tranin stability and data invariance, Half Instance Normalization block is created. The main advantage compared to batch normalization is that HIN block maintains its normalization procedure while on both training and inferencing. The architecture of HIN block compared to Res Block is like the image above. HIN block resembles Res block a lot, but in HIN Block, the section for half instance normalizing exists.
Loss Function
For the loss function, researchers used simple negatived PSNR metric which is aggregated for both outputs from stage 1 and 2. PSNR is a metric used frequently in image/video denoising or restoration. The Loss function in a written form is like above.
Ablations and Experiments
Researchers have tested various changes to HINet including changing the normalization method which is the main feature of HINet. The result was as we may assume, every other normalization methods made the performance decrease which is measured by PSNR.
HINet with half instance normalization was compared to various models on datasets like SIDD, REDS, GoPro and Rain13k, and HINet became the new SOTA for those tasks. Moreover, HINet had much shorter inference time than the previous SOTA model named MPRNet on those four datasets.
Conclusion
The reason why HINet is meaningful is that it not only surpassed the previous SOTA but it tried to use normalization method in image restoring and produced successful results. This result could stimulate other researchers to create other astonishing normalization methods and apply them to super resolution tasks.