Loading page content
Deep Learning Research·2025
Interactive Ensemble Architecture: Moving Beyond Late Fusion
Taking the DIAE concept further by breaking the walls between models. A novel architecture where models collaborate during feature extraction — not just at the end.
Deep LearningInnovationComputer VisionEnsemble Learning
Role & Outcomes
- Extending the DIAE work into interactive ensembles where backbones share features mid-flight.
- Prototyped cross-stitch and cross-attention bridges with competence-aware routing to boost robustness.
- Capturing learnings in public notes and code so the next iteration is faster and more measurable.
The Evolution: From DIAE to Interactive
In the BSc graduation project, DIAE (Dynamic Input-Adaptive Ensemble) successfully demonstrated that we can significantly improve performance by dynamically weighting models based on the input.
However, even in DIAE, the models (ResNet, DenseNet, EfficientNet) were still working in isolation. They extracted features separately, and we only combined their wisdom at the very end.
"Why not extract the extracted data? Why can't they help each other while they are working?"
The Innovation: Breaking the Walls
The next step was to design an Interactive Ensemble Architecture that enables real-time information exchange during feature extraction.
Instead of just a "captain" picking the best player, the players now pass the ball to each other during the game. If ResNet detects a strong edge, it can "tell" DenseNet, which might use that context to better recognize cellular patterns.
Architecture: The Two-Pass Strategy
01
Intra-Backbone Coupling
Cross-Stitch Units and Cross-Attention Bridges mix features at multiple network depths. This allows feedback to be injected into each backbone before its next stage.
02
Competence-Aware Routing
Similar to DIAE, a router is used — but this router predicts coupling strengths per-stage. It decides how much collaboration is needed at each depth of the network.
Ongoing Research
This is an ongoing research project aimed at solving the "Sequential Bottleneck" of traditional ensembles. By enabling models to "see" what others discover, the goal is to move towards a truly collaborative AI system.
Early validation shows promising results on the test set, outperforming traditional late fusion by 2–3%. The goal is to refine this architecture to handle even more complex histopathology cases where single models struggle.