My opinion on critiquing LLMs for education with the argument that they are just mimicking words in the training data: > I believe that generally people underestimate the capabilities of emergent complexity. It depends on your philosophical position, particularly in whether you subscribe to physicalism and that intelligent is simply produced through the interaction of matter in complex ways. Whilst we can say LLMs are simply predictive models, simple rules at scale can produce sophisticated behaviours. Take ants for example, they operate with very simple rules but in numbers the behaviours lead to very interesting results: [see ant colony optimisation algorithms](https://en.wikipedia.org/wiki/Ant_colony_optimization_algorithms). Hence, even if the objective of the training is related to prediction, qualitative changes in cognitive capacity can occur at particular model sizes (kind of like [[Piaget's theory of cognitive development|Piaget's stages of development]]). We see that LLMs are developing internal models of space and time, it may also be developing models of reasoning in the pursuit of better prediction. Though of course, we can argue anything if we change the semantics of 'intelligence' enough, would logic programming languages be considered intelligence? In a sense yes, but nothing like human thinking. Though there is a danger in assuming a likeness in intelligence between LLMs and humans that has real consequences for education. Would we really want an intelligence without empathy, teaching our kids how to feel and be empathetic?