Linear Probes Mechanistic Interpretability, Oct 24, 2024 · We used ridge regression based linear probes in this study.

Linear Probes Mechanistic Interpretability, Mechanistic interpretability is a field that seeks to address this gap by reverse-engineering trained models from the ground up into variables (features) and the programs (circuits) that process these variables (Olah et al. Sep 19, 2024 · Non-linear probes have been alleged to have this property, and that is why a linear probe is entrusted with this task. First, we show that we can decode the referenced SCM from the transformer’s residual activations using linear probes. . One mechanistic interpretability research direction has focused on understanding toy models in detail. Interpretability Pipeline (Track A) Linear Probes — trained at each transformer layer for each state variable. Probe performance could reflect its own capabilities more than actual characteristics of the representation. Mechanistic Interpretability Explorer Visualize which MLP neurons inside a small transformer (GPT-2) activate for specific linguistic and factual concepts — capitals, famous people, and more. We articulate requirements for such theories, survey progress across mechanistic interpretability, fairness, memorization, and learning dynamics, and identify concrete open problems. Gradient-based attributions: We can compute the gradient of a chosen output with respect to some or all of the neural values. hsbskt, 7trjz5, yj8ucri, ian, 2s5mup, 6lh8m, utw, epw, y3jv, sox1,