Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Chapter 3: Multiagent Search - Implementation

Complete Python implementations of game-playing algorithms with interactive exploration.

Contents

  1. Framework

  2. Minimax

  3. Alpha-Beta Pruning

  4. MCTS

  5. Tic-Tac-Toe

  6. Connect Four

  7. Comparisons

  8. Interactive Playground

  9. Guided Mini-Labs

  10. Summary

1. Framework {#framework}

This section defines a common interface used by all game environments in the notebook. A shared interface is important because it lets us plug different games into the same search algorithms without rewriting minimax, alpha-beta, or MCTS each time.

What the framework standardizes

  • get_legal_actions(player): returns valid moves.

  • get_successor(action): returns the next game state after a move.

  • is_terminal(): checks whether the game is over.

  • utility(player): returns payoff from a player’s perspective.

  • current_player(): indicates whose turn it is.

After running the next two code cells, every algorithm in later sections can work with any class that follows this interface.

Libraries imported (including ipywidgets for interactivity)
GameState framework defined

2. Minimax {#minimax}

Minimax is the baseline adversarial search method for deterministic two-player zero-sum games. It assumes both players act optimally.

Why this implementation matters

  • It uses a fixed root-player perspective so returned values are consistent.

  • It supports optional evaluation functions for depth-limited search.

  • It tracks metrics (nodes, cutoff_evals) to make complexity visible.

When you run demonstrations later, compare minimax metrics against alpha-beta to understand pruning benefits.

Minimax implemented with fixed-perspective scoring and metrics

3. Alpha-Beta Pruning {#alphabeta}

Alpha-beta pruning is an optimization of minimax that preserves the same final decision while exploring fewer nodes.

What to look for in this code

  • Same game-theoretic objective as minimax.

  • Bound tracking with alpha and beta.

  • Instrumentation for prunes so speedups are measurable.

In later test cells, focus on node count reduction and runtime improvement relative to minimax at the same depth.

Alpha-Beta implemented with fixed-perspective scoring and metrics

4. MCTS {#mcts}

Monte Carlo Tree Search (MCTS) is useful when full search is too expensive or heuristic design is difficult. Instead of exhaustive expansion, it estimates promising actions via repeated simulations.

Implementation highlights

  • UCT-based child selection balances exploration and exploitation.

  • Random rollouts estimate outcomes from frontier states.

  • Backpropagation accumulates statistics at visited nodes.

  • Metrics (simulations, expansions, tree_nodes) show search effort.

Unlike minimax and alpha-beta, MCTS can improve progressively as you increase the simulation budget.

MCTS implemented with consistent root perspective and metrics

5. Tic-Tac-Toe {#tictactoe}

Tic-Tac-Toe is a compact environment for validating algorithm correctness. Because the state space is small, it is ideal for comparing complete-search methods and simulation methods in a controlled setting.

Why it is useful pedagogically

  • Rules are simple and transparent.

  • Minimax can often search deeply enough for near-complete analysis.

  • Differences in runtime and node counts are easy to observe.

After defining the game class and evaluation function, the mini-lab demonstrates performance behavior for each algorithm.

Tic-Tac-Toe and ttt_eval implemented

Mini-Lab A: Depth vs Search Cost (Tic-Tac-Toe)

Hypothesis: deeper search improves decision quality but increases search cost. This lab measures that tradeoff directly.

How to read the output

  • Minimax nodes: total nodes visited without pruning.

  • Alpha-Beta nodes: total visited with pruning.

  • Alpha-Beta prunes: number of cut branches.

Discussion prompt

At equal depth, does alpha-beta preserve the same decision while reducing explored nodes? If yes, by how much as depth grows?

=== Tic-Tac-Toe Metrics Demo ===
Initial:
. . .
. . .
. . .

Minimax (depth 9):
Best move: (0, 0), value: 0.00, time: 14.6891s
Nodes: 549946, Cutoff evals: 0

Alpha-Beta (depth 9):
Best move: (0, 0), value: 0.00, time: 0.6695s
Nodes: 18297, Cutoff evals: 0, Prunes: 8180

MCTS (1000 sims):
Best move: (1, 1), time: 0.3795s
Simulations: 1000, Expansions: 1000, Tree nodes: 1001

6. Connect Four {#connect4}

Connect Four has a larger branching factor and deeper tactical dependencies than Tic-Tac-Toe, making it a more realistic stress test for search algorithms.

Why this section is important

  • Complete minimax search is usually infeasible from the initial state.

  • Depth-limited alpha-beta with a heuristic becomes necessary.

  • MCTS is often competitive because it allocates effort adaptively.

Use this section to see how algorithm behavior changes when the game becomes more complex.

Connect Four implemented

Evaluation Function for Connect Four

This heuristic estimates state quality from the root player’s perspective.

Scoring intuition

  • Reward center control because central columns create more 4-in-a-row opportunities.

  • Reward 2-in-a-row and 3-in-a-row patterns with open extension spaces.

  • Penalize opponent threats to encourage blocking behavior.

The heuristic is intentionally simple so students can modify weights and observe strategy changes.

Evaluation functions defined for Tic-Tac-Toe and Connect Four

Mini-Lab B: Simulations vs Stability (MCTS)

Hypothesis: increasing simulations improves stability of selected actions and confidence.

How to run this lab

  • Keep the same initial state.

  • Run repeated MCTS calls at one simulation budget.

  • Increase simulations and compare action frequency concentration.

A more concentrated action distribution at higher budgets suggests lower variance in search estimates.

=== Connect Four Metrics Demo ===
Initial:
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
0 1 2 3 4 5 6

Alpha-Beta (depth 4):
Best column: 3, value: 4.00, time: 0.2064s
Nodes: 737, Cutoff evals: 540, Prunes: 137

MCTS (500 sims):
Best column: 4, time: 2.0687s
Simulations: 500, Expansions: 500, Tree nodes: 501

7. Comparisons {#comparisons}

This section summarizes practical algorithm tradeoffs after the experiments.

What to compare

  • Decision quality under the same compute budget.

  • Runtime and node/simulation counts.

  • Sensitivity to heuristic quality and parameter settings.

Use these comparisons to justify algorithm choice by problem characteristics, not by preference.

=== Algorithm Comparison ===

Tic-Tac-Toe:
  Minimax: Complete search, slow
  Alpha-Beta: same move as Minimax, ~30x fewer nodes (18,297 vs 549,946)
  MCTS: Fast, good for learning

Connect Four:
  Alpha-Beta + eval: Limited depth (4-6 plies)
  MCTS: Handles large branching factor

Key Insights:
  - Alpha-Beta essential for deep search
  - MCTS great when eval function hard to design
  - Modern: Combine both (AlphaZero)

8. Interactive Playground {#interactive-playground}

Use this playground to run controlled experiments without editing code manually.

Controls

  • Game: switch between Tic-Tac-Toe and Connect Four.

  • Algorithm: choose Minimax, Alpha-Beta, or MCTS.

  • Depth / Sims / UCT c: tune search horizon and simulation behavior.

Suggested experiments

  • Compare Minimax vs Alpha-Beta at equal depth.

  • Increase MCTS simulations and observe action stability.

  • Test Connect Four with shallow vs deeper search to see heuristic effects.

Each run reports the chosen action, metrics, and resulting next state so decisions are easy to inspect.

Loading...
Loading...

Play Against the AI (Tic-Tac-Toe)

Play as X against an AI opponent to build intuition about search behavior.

How this helps learning

  • You can feel the difference between deterministic search (Alpha-Beta) and simulation-based play (MCTS).

  • Parameter changes immediately affect move quality and style.

  • Game outcomes provide fast feedback on algorithm settings.

Try multiple rounds with different settings and note when AI choices change.

Loading...

9. Guided Mini-Labs {#guided-mini-labs}

These mini-labs connect theory to observable behavior through short, repeatable experiments.

Learning goals

  • Relate depth to computational cost.

  • Relate simulation budget to action stability.

  • Validate conceptual understanding with quick checks.

Run labs in order for the clearest progression: cost analysis, stability analysis, then concept verification.

Loading...
Loading...
Loading...

10. Summary {#summary}

This notebook combines algorithm implementation, measurable diagnostics, and interactive learning activities.

What you implemented

  • Minimax with fixed root-player perspective and instrumentation

  • Alpha-Beta pruning with node and prune metrics

  • MCTS with simulation and tree-growth metrics

What you explored

  • Runtime and search-cost tradeoffs across algorithms

  • Heuristic-based depth-limited play in Connect Four

  • Interactive parameter tuning and human-vs-AI gameplay

What to take away

  • Alpha-Beta can match minimax decisions with far less search.

  • MCTS quality generally improves with larger simulation budgets.

  • Good evaluation functions are critical in large game spaces.

  • Algorithm choice should follow problem structure and compute budget.

For theory, see: Chapter 3: Multiagent Search