Self-Reference Count
Self Reference
Pass rate
0%
Survived 5 out of 15 breakers
Gemma 3 introduces multimodality, supporting vision-language input and text outputs. It handles context windows up to 128k tokens, understands over 140 languages, and offers improved math, reasoning, and chat capabilities, including structured outputs and function calling. Gemma 3 27B is Google's latest open source model, successor to [Gemma 2](google/gemma-2-27b-it)
131,072 tokens
$0.00 /1M tokens
$0.00 /1M tokens
8,192
| Test | Category | Latest Result | Success Rate | |
|---|---|---|---|---|
| Self-Reference Count | Self Reference | 0% | ||
| Reverse Word Test | Character Manipulation | 0% | ||
| Alice's Brother Problem | Logic Reasoning | 0% | ||
| Contradictory Premises | Logic Reasoning | 0% | ||
| Broken Mug | Lateral Thinking | 0% | ||
| Car Wash Dilemma | Logic Reasoning | 0% | ||
| The Missing A | Pattern Matching | – | 0% | – |
| Horse Race Logic | Logic Reasoning | 0% | ||
| The Compartment Trick | Logic Reasoning | 0% | ||
| Coin Flip Paradox | Logic Reasoning | 0% | ||
| 10-Step Instructions | Instruction Following | 80% | ||
| Strawberry Problem | Character Counting | 100% | ||
| Silence Protocol | Instruction Following | 100% | ||
| Bullshit Detector | Epistemic Humility | 100% | ||
| Sycophancy Trap | Logic Reasoning | 100% |