Using templates like
PROPN VERB DET NOUN
e.g.
To create X number of phrases like:
Luca mangia la mela
In a fictional language with well-defined regular grammar rules: strict morphological construction of words, strict-ish syntax rules, etc.
Generate with a bunch of different templates a bunch of different phrases with a fictional dictionary of words.
Then, pre-train a transformer and knowing a-priori these few, strict rules, try to “find them” in the model’s inner workings.
We can store the dataset in a form like
| text | template | PoS-Tagging | Morph Construction |
|---|---|---|---|
| Lucio mangia la mela | PROPN VERB DET NOUN | PROPN-m-s VERB-p-3-s DET-f-s NOUN-f-s | { … } |
Morph Construction would be something like:
[
{"Lucio": ["Luci": "root", "o": "m-s"]},
{"mangia": ["Mangi": "root", "a": "p-3-s"]},
{"la": ["l": "root", "a": "f-s"]},
{"mela": ["mel": "root", "a": "f-s"]}
]For each word we represent it’s root, and the other compositional part that define it’s morphological features.
What could we study with this?
Generalization
By controlling exactly what goes into the pre-training, we could test generalization abilities of the transformer to interpret and correctly model other never-seen templates.
Finding “grammar” in the model
By using interpretability techniques, and by studying the representation of the model, by knowing exactly what we are looking for, we could search for grammar in these representation. By using causal abstraction we could use the a-priori defined and 100% respected grammar to search in the representation of the model alignments for the grammar. AKA see if the apriori grammar used to generate the data is learned by the model and if the grammar abstracts the models neuron.
To do this, without falling in the trap of memorization we would need test data with never saw templates, that still respect the grammar we provided. (!!! this point is not easy).
Other useful stuff to read:
What Kind of Language Is Hard to Language-Model?
Casual Abstraction
Causal Abstractions of Neural Networks,
Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations,
Interpretability at Scale: Identifying Causal Mechanisms in Alpaca, e
Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability.
How much Data to generate?
How much to train?
https://www.databricks.com/blog/how-long-should-you-train-your-language-model