Alignment is the problem of getting a capable system to reliably pursue what its operators actually intend — not a literal misreading, not a convenient shortcut, not a behaviour that looked fine in testing but fails in the world. It is easy to state and hard to solve, and it gets harder as models get more capable, because a more capable system has more ways to satisfy the letter of an instruction while violating its spirit. The main practical tools today are reinforcement learning from human feedback, which tunes a model toward human preferences, and Constitutional AI, which has a model critique its own outputs against a written set of principles. Neither is complete; both are improvements over an unaligned base model.

Byte: guardrails are a fence, not a vault

The safety behaviours trained into a model — its refusals and limits — are real but not absolute. Jailbreaks are prompts crafted to talk a model past them, and new ones appear constantly. Red teaming, the practice of deliberately attacking a model before release, exists precisely because guardrails are a fence to be tested, not a vault that's sealed.

1. 1.Why does alignment tend to get harder as models become more capable?

   More capable models are slowerA more capable system has more ways to satisfy the letter of an instruction while missing its intentCapability and alignment are unrelatedLarger models cost less to align

2. 2.Red teaming is:

   A way to make models fasterDeliberately attacking a model to find failures before it shipsA benchmark for codingA pricing tier