My First Big GNN Research Project: Lessons from Oversmoothing

This oversmoothing project was my first major research experience in AI, and it was one of the most valuable learning experiences I’ve had so far. I went into it interested in graph neural networks, but I came out with a much deeper understanding of how to actually do effective research: define clear questions, reproduce baselines carefully, run controlled extensions, and document findings with discipline.

The core problem we studied was oversmoothing in deep GNNs — the tendency for node representations to become indistinguishable as depth increases. I started by reproducing key results from Untrained GNN Tickets (UGTs), then extended the analysis with additional methods like Unified Graph Sparsification (UGS), Weight Reparameterization (WeightRep), and initialization studies. Working through these comparisons helped me understand how to move from “interesting intuition” to evidence-backed conclusions.

One of the most important technical insights for me was that sparsity alone is not a complete fix. In our experiments, sparse models that still trained weights could collapse, while untrained sparse subnetworks remained stable much deeper. That contrast made the broader point very clear: oversmoothing is deeply tied to optimization dynamics, not only to architecture depth or graph structure.

Beyond the technical side, this project taught me how research actually gets done day to day. I had a supervisor I met with regularly, and those meetings were incredibly helpful. They guided me on how to frame hypotheses, design stronger experiments, and pressure-test interpretations before claiming conclusions. That mentorship made a huge difference in helping me develop better research habits and confidence.

I also learned to value reproducibility and diagnostic tooling. Tracking metrics like MAD, validating trends across datasets and architectures, and carefully comparing methods under consistent protocols were all essential. The process was sometimes slow, but it showed me that strong research is less about one flashy result and more about consistent, rigorous reasoning.

Overall, this project gave me both practical GNN depth and a foundation in research methodology. It confirmed that I genuinely enjoy research work, especially when it combines theory, experimentation, and iterative feedback from mentorship. As my first big research project, it set the standard for how I want to approach future AI research.