EP 316 Ken Stanley on the AI Representation Problem

Jim talks with Ken Stanley about the Fractured Entanglement Representation hypothesis in deep learning neural networks. They discuss open-endedness in AI systems & evolution, the Picbreeder experiment & its significance, the objective paradox of finding things by not looking for them, comparisons between Picbreeder & SGD networks, visual differences in internal representations, weight sweep experiments, modular vs tangled decomposition, implications for creativity & continual learning & generalization abilities, Unified Factored Representation as an alternative to FER, the relationship to grokking in neural networks, scaling considerations & evidence in larger models, potential methods to achieve UFR, connections to biological evolution and DNA representation, and much more.