Validation in AI Era

---

# 🔍 Validation in AI Era

@luk036 👨‍💻 · 2026 📅

---

## 🤔 The Core Problem

---

### 🤖 AI ≠ Reasonable

- AI generates code that compiles and runs
- Human: It looks reasonable. ✅
- But AI has no concept of Reasonable

- The solution: automated validation 🛡️

---

### 🛡️ Core Principle

- If you can't validate, **don't change** 🚫
- e.g. "Recursive gcd is slower than iterative gcd?" → Prove it!

---

### ⚡ The Golden Rule

**Either modify source code, OR unit tests.**

**Never both at the same time.**

]

- If the API contract changes 📜
  - At least keep the **numerical values** of unit tests unchanged
  - Let the tests tell you what broke

---

### 🔄 Fix-then-Validate Loop

.mermaid[
<pre>
graph LR
    A["📝 Source"] --> B["✏️ Modify"]
    B --> C["⚙️ /fix-srt"]
    C --> D["✅ /validate-srt"]
    D -- "Pass ✅" --> E["🚀 Done"]
    D -- "Fail ❌" --> F["🔧 Investigate"]
    F --> B

style A fill:#e3f2fd,stroke:#1565c0,color:#2e3440,stroke-width:3px
    style B fill:#e8eaf6,stroke:#283593,color:#2e3440,stroke-width:3px
    style C fill:#fff9c4,stroke:#f57f17,color:#2e3440,stroke-width:3px
    style D fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
    style E fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
    style F fill:#ffcdd2,stroke:#c62828,color:#2e3440,stroke-width:3px
</pre>
]

> 🔑 Always fix, then validate. Never skip validation.

---

## 🔄 Round-trip Tests

---

### 🔄 What are Round-trip Tests?

- Apply a transformation ${\color{steelblue}T}$
- Then apply its **inverse** ${\color{purple}T^{-1} }$
- Result must equal the original input

$${\color{purple}T^{-1} }({\color{steelblue}T}(x)) = x$$

- AI can generate both directions — validate they truly invert ⚡

---

### 🔄 Round-trip Test Examples

$$
\begin{aligned}
{\color{purple}\texttt{decode}}({\color{steelblue}\texttt{encode}}(x)) &= x \\[4pt]
{\color{purple}\texttt{inv\_spectral\_fact}}({\color{steelblue}\texttt{spectral\_fact}}(r)) &= r \\[4pt]
{\color{purple}\texttt{poly}}({\color{steelblue}\texttt{roots}}(p)) &= p \\[4pt]
{\color{purple}\texttt{to\_decimal}}({\color{steelblue}\texttt{to\_csd}}(n)) &= n \\[4pt]
{\color{steelblue}p} + v - v &= {\color{steelblue}p}
\end{aligned}
$$

---

## 📐 Property-based Tests

---

### 📐 Algebraic Properties

These must hold for **any** valid input:

$$
\begin{aligned}
{\color{steelblue}x} &= {\color{steelblue}x} && \text{(reflexivity)} \\[4pt]
{\color{steelblue}x} \cdot {\color{purple}y} &= {\color{purple}y} \cdot {\color{steelblue}x} && \text{(commutativity)} \\[4pt]
{\color{steelblue}x} \cdot ({\color{purple}y} \cdot {\color{darkcyan}z}) &= ({\color{steelblue}x} \cdot {\color{purple}y}) \cdot {\color{darkcyan}z} && \text{(associativity)} \\[4pt]
{\color{steelblue}x} \cdot ({\color{purple}y} + {\color{darkcyan}z}) &= {\color{steelblue}x} \cdot {\color{purple}y} + {\color{steelblue}x} \cdot {\color{darkcyan}z} && \text{(distributivity)}
\end{aligned}
$$

> 🔑 If AI implements an operator, verify these on random inputs!

---

### 📐 Geometric Properties

For any point set $P$:

$$
\begin{aligned}
\texttt{area}(\texttt{convex\_hull}(P)) &\ge \texttt{area}(P) \\[4pt]
\sum \texttt{area}(\texttt{polygon\_cut}(P)) &= \texttt{area}(P) \\[4pt]
\texttt{total\_dual\_cost} &\le \texttt{total\_primal\_cost}
\end{aligned}
$$

- Area of convex hull ≥ area of original shape 🏗️
- Polygon partitioning preserves total area ✂️
- Duality: upper bound ≥ optimal value ⚖️

---

### 📐 Graph Properties

For any graph $G$ and point set $\textit{ps}$:

$$
\begin{aligned}
\texttt{is\_bipartite}(\texttt{max\_cut}(G)) &= \text{true} \\[4pt]
\texttt{is\_bipartite}(\texttt{odd\_cycle\_cover}(G)) &= \text{true} \\[4pt]
\texttt{is\_hamiltonian\_cycle}(\texttt{tsp}(\textit{ps})) &= \text{true}
\end{aligned}
$$

- Max cut must yield a bipartite subgraph 🔵🔴
- Odd cycle cover must produce bipartite result
- TSP output must be a valid Hamiltonian cycle 🛤️

---

### 🛠️ Libraries for Property-based Tests

```python
from hypothesis import given, strategies as st

@given(st.integers(), st.integers())
def test_commutative(a, b):
    assert a + b == b + a
```
]

```rust
use proptest::prelude::*;

proptest! {
    #[test]
    fn test_commutative(a: i32, b: i32) {
        assert_eq!(a + b, b + a);
    }
}
```
]

**⚙️ C++ — RapidCheck**

```cpp
#include <rapidcheck.h>

rc::check("addition is commutative",
    [](int a, int b) {
        RC_ASSERT(a + b == b + a);
    });
```

> 🔑 Frameworks auto-generate random inputs — let the fuzzer find edge cases!

---

### 🧪 Test Strategies Comparison

.mermaid[
<pre>
graph TD
    A["🧪 Test Strategies"] --> B["🔄 Round-trip"]
    A --> C["📐 Property-based"]
    B --> B1["Encode/Decode"]
    B --> B2["Transform/Inverse"]
    B --> B3["Add/Subtract"]
    C --> C1["Algebraic Laws"]
    C --> C2["Geometric Invariants"]
    C --> C3["Graph Constraints"]

style A fill:#fff9c4,stroke:#f57f17,color:#2e3440,stroke-width:3px
    style B fill:#e3f2fd,stroke:#1565c0,color:#2e3440,stroke-width:3px
    style C fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
    style B1 fill:#e3f2fd,stroke:#1565c0,color:#2e3440,stroke-width:3px
    style B2 fill:#e3f2fd,stroke:#1565c0,color:#2e3440,stroke-width:3px
    style B3 fill:#e3f2fd,stroke:#1565c0,color:#2e3440,stroke-width:3px
    style C1 fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
    style C2 fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
    style C3 fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
</pre>
]

---

### 🎯 Putting It All Together

.mermaid[
<pre>
graph LR
    A["🤖 AI Code"] --> B["👀 Review"]
    B --> C["🧪 Write Tests"]
    C --> D["🔄 Round-trip"]
    C --> E["📐 Properties"]
    D --> F{"✅ Pass?"}
    E --> F
    F -- "Yes ✅" --> G["✅ Accept"]
    F -- "No ❌" --> H["🔧 Fix"]
    H --> A

style A fill:#e3f2fd,stroke:#1565c0,color:#2e3440,stroke-width:3px
    style B fill:#e8eaf6,stroke:#283593,color:#2e3440,stroke-width:3px
    style C fill:#fff9c4,stroke:#f57f17,color:#2e3440,stroke-width:3px
    style D fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
    style E fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
    style F fill:#ffe0b2,stroke:#e65100,color:#2e3440,stroke-width:3px
    style G fill:#c8e6c9,stroke:#2e7d32,color:#2e3440,stroke-width:3px
    style H fill:#ffcdd2,stroke:#c62828,color:#2e3440,stroke-width:3px
</pre>
]

---

# Questions? 💬