01 / Research

Restoring what is missing.
Protecting what remains.

A research project on high-fidelity mural restoration, pairing mask-aware local texture modeling with long-range structural reasoning.

Read the paper

High-Fidelity Mural Restoration via a Unified Hybrid Mask-Aware Transformer

Ancient murals rarely fail in just one way. Fine cracks coexist with large areas of peeling; surviving pigment must remain untouched even as lost structure is reconstructed. HMAT treats these as one connected restoration problem.

A

Background

Environmental exposure, material aging, biological decay, and human activity leave murals with mixed damage at very different scales. Methods built for ordinary photographs often miss the distinct line work, palette, and historical significance of mural imagery.

B

The challenge

A useful restoration must infer broad, coherent forms across large losses while keeping the texture of surviving paint sharp. It must also preserve every authentic pixel outside the damaged mask instead of shifting color or inventing detail around it.

HMAT framework showing the hybrid encoder, mask-conditional style fusion, transformer, decoder, and refinement network
Figure 01 HMAT combines a mask-aware encoder, Transformer bottleneck, mask-conditioned style fusion, and two-stage synthesis.
How it works

One pipeline, three kinds of reasoning

  1. 01

    Read the boundary

    Mask-Aware Dynamic Filtering changes the local kernel according to nearby damage, limiting contamination from zero-filled holes and retaining fine boundary detail.

  2. 02

    Connect distant structure

    A Transformer bottleneck borrows context from valid mural regions so lines, figures, and composition can continue across wide missing areas.

  3. 03

    Adapt to the damage

    Mask-conditioned style fusion combines image semantics, latent variation, and mask geometry. Thin cracks and large losses therefore receive different synthesis strategies.

Selected results

Strongest gains under severe damage

HMAT was tested on DHMural and a Nine-Colored Deer dataset using identical fixed masks for every method. Moderate masks cover 20–30% of the image; severe masks cover 40–50%.

26.05PSNR ↑
0.857SSIM ↑
17.40FID ↓
0.022L1 ↓

Nine-Colored Deer · severe mask · best reported result across all four metrics in the controlled comparison.

Qualitative DHMural comparison between masked input, MADF, MAT, HMAT, and ground truth
HMAT preserves long contours and local texture more consistently than the compared baselines.
Ablation comparison of style-fusion capacity configurations
The selected style-capacity balance reduces color shift and boundary artifacts.
What mattered most?The fidelity-oriented training objective produced the largest ablation gain; MADF encoding and mask-guided style conditioning were the next main contributors.
Current limits. Experiments use 256 × 256 inputs; the Nine-Colored Deer set comes from one mural; masks are supplied rather than automatically detected; ambiguous losses may still need expert review.
Restoration tool

A research model made legible

The HMAT web interface turns the restoration workflow into a guided visual sequence. Visitors can upload a damaged mural or open a sample, compare HMAT with other restoration approaches, inspect image-quality metrics, and save outputs for closer review.

The interface keeps the original image, mask, model results, and evaluation measures in view together—useful for explaining what the model changes, and just as importantly, what it leaves untouched.

Tool website · link forthcoming