Verbal Analogy

Verbal Analogy – Advanced Study Notes

Course : Mental Ability Test (MAT) – NTSE
Level : MIT‑style, first‑principles, rigorous reasoning
Audience : Graduate students in Mathematics, Computer Science, Linguistics
Goal : Master the theory, algorithmic models, and edge‑case handling of verbal analogies.

1. Introduction

A verbal analogy is a linguistic expression of a relation between two pairs of words or phrases:

A : B :: C : D

It asks: What is the relationship between C and D given the relationship between A and B?
In the typical MAT / NTSE format, candidates are presented with four words (A, B, C, D) and must select the best completion of the analogy.

Despite its apparent simplicity, the analogy problem is a rich research area intersecting:

• Logic & Reasoning
• Formal Semantics
• Cognitive Science
• Natural Language Processing (NLP)
• Machine Learning

2. Formal Foundations

2.1 Definition

Let ( V ) be a finite or infinite set of lexical items (words, multi‑word expressions).
Define a binary relation ( R \subseteq V \times V ).
An analogy is a quadruple ( (a,b,c,d) \in V^4 ) satisfying:

[ R(a,b) ;\wedge; R(c,d) ]

In classic analogical reasoning, we require that ( R ) be consistent across the two pairs: the type of relation ( R ) (e.g., is_the_capital_of, is_synonym_of, is_antonym_of, etc.) is the same.

2.2 Types of Relations

Key Insight: In analogies, the relation is often transitive: if ( R(a,b) ) and ( R(c,d) ) hold, then ( R(a,c) ) and ( R(b,d) ) might also hold under certain conditions.

2.3 Logical Viewpoint

Treat each word as a predicate or constant in a first‑order language ( \mathcal{L} ).
Define a binary predicate ( \text{REL}(x,y) ) capturing the relation.

An analogy task asks whether:

[ \exists \text{REL} ; :; \text{REL}(a,b) ;\wedge; \text{REL}(c,d) ]

The challenge is to infer or learn ( \text{REL} ) from sparse data.

3. Theoretical Models

3.1 Relational Algebra

Let ( \mathbf{E} ) be the set of all possible entity embeddings (vectors).
Define a relation operator ( \mathcal{R} : \mathbf{E} \to \mathbf{E} ) such that:

[ \mathbf{e}_b = \mathcal{R}(\mathbf{e}_a) ]

If ( \mathcal{R} ) is linear (matrix multiplication) or translational (vector addition), many NLP models use it.

3.1.1 TransE (Bordes et al., 2013)

[ \mathbf{e}_a + \mathbf{r} \approx \mathbf{e}_b ]

Where ( \mathbf{r} ) is a relation vector. The analogy reduces to:

[ \mathbf{e}_c + \mathbf{r} \approx \mathbf{e}_d ]

Thus, solving analogies is vector arithmetic.

3.1.2 Bilinear & Holographic Models

More expressive models:

[ \langle \mathbf{e}_a, \mathbf{W}_r \mathbf{e}_b \rangle ]

or holographic reduced representations using circular convolution.

3.2 Distributional Semantics & Contextual Embeddings

Modern analogies use contextual embeddings (BERT, RoBERTa).
Define the contextual embedding ( \mathbf{e}_w^{(c)} ) as the vector for word ( w ) in context ( c ).

• For word‑in‑definition analogies, we can compute the relation vector as:

[ \mathbf{r} = \frac{1}{|C|}\sum_{c\in C}\left(\mathbf{e}_b^{(c)} - \mathbf{e}_a^{(c)}\right) ]

• For cross‑sentence analogies, use sentence embeddings.

3.3 Graph‑Theoretic Perspective

Represent lexical knowledge as a graph ( G = (V,E) ).
Edges ( (u,v) ) are labeled by relation types.
An analogy corresponds to a commutative diagram:

  A ----R----> B
  |            |
  R            R
  |            |
  v            v
  C ----R----> D

The commutativity condition: traversing along edges yields the same relation.

3.4 Cognitive Models

• Structure Mapping Theory (Gentner, 1983): mapping is based on relational structure rather than attribute similarity.
• Proto‑typicality & Prototype Matching: analogies can involve typical instances of a relation.

4. Edge Cases & Pitfalls