Joint Self-Supervised Compression Method for ARC-AGI Problem Solving

Explore a controversial compression method that achieved 27.5% performance on ARC-AGI v1 using only $2 worth of compute through an in-depth interview with the method's author Mithil Vakde. Dive into the fundamentals of joint self-supervised compression and understand how this approach differs from traditional methods like HRM, TRM, and CompressARC. Learn about the training flow, data augmentation challenges, and the theoretical foundations connecting minimum description length (MDL) principles to practical AI problem-solving. Examine the architectural choices including 3D RoPE implementation, scaling considerations, and the potential for this method to work as a general problem solver beyond the ARC-AGI benchmark. Discover insights about explicit versus implicit MDL approaches, test-time training strategies, and the author's perspective on why compression methods are underexplored in this domain. Gain understanding of the method's applicability to ARC-AGI v2, scaling challenges, and future directions for removing data augmentation dependencies while maintaining performance efficiency.

Syllabus

- Intro:
- overview of the method:
- related works HRM/TRM/CompressARC:
- method overview:
- interview with author:
- background of mithil:
- overview of the intuition:
- training flow of the method:
- data augmentation is bad:
- why are you so interested by ARC?:
- does the method work with ARC AGI v2?:
- why so few compression method?:
- what is joint self supervised learning?
- explicit vs implicit mdl:
- connection between mdl and TRM?:
- is this method a general problem solver?:
- is compression enough for this benchmark?:
- architecture choices:
- how much does the 3D RoPE helps?:
- how to scale this?:
- dumbest idea ever:
- how to remove data augmentation?:
- about test time training:
- Conclusion: