Generating sequences

The question: what do sequences from this population look like, and can I get new ones to use as a background or a null set?

Drawing sequences

sample draws sequences in proportion to their probability: a sequence twice as probable is drawn twice as often. Give length to draw only sequences of that length.

seed chooses the draw: the same seed always gives the same sequences, and leaving it out uses seed 0, so two calls without a seed return the same draws. Pass different seeds for independent draws, for example two null sets.

sample.py

import BEAM

model = BEAM.load_pretrained("human_tcrb_cdr3_aa_foundation")

print(model.sample(5, seed=1))             # any length
print(model.sample(5, length=12, seed=1))  # exactly 12 residues

Output

['CASSPGTSSDTYEQYF', 'CASSLGTYSYNEQFF', 'CASRPGNSPLHF', 'CSVLKSGTPGEKLFF', 'SVGSGVNIGEQYF']
['CALLTENYGYTF', 'CASSLFDRGGYE', 'CASNHSNQPQHF', 'CASSVDRGEQYV', 'CSAPRGFDGYTF']

Full program, including the set-up hidden here

Sequences with their probabilities

simulate draws the same way and also returns each sequence’s log probability, which saves scoring them afterwards.

simulate.py

import BEAM

model = BEAM.load_pretrained("human_tcrb_cdr3_aa_foundation")
draws = model.simulate(4, seed=3)

for cdr3, logp in zip(draws.sequences, draws.log_probs):
    print(f"{cdr3:20} log P = {logp:.3f}")

Output

CSARQSSYEQYF         log P = -14.110
CASSSNRVRTGELFF      log P = -18.039
CASSPGPIKNTGELFF     log P = -19.780
CASSLGTSGDHRELFF     log P = -22.735

Full program, including the set-up hidden here

The draws follow the model

Draw many sequences and their lengths follow the model’s length distribution, as they should:

lengths.py

from collections import Counter
import BEAM

model = BEAM.load_pretrained("human_tcrb_cdr3_aa_foundation")
draws = model.sample(50_000, seed=11)

observed = Counter(len(s) for s in draws)
expected = model.length_distribution
print("length  sampled share  model probability")
for L in range(10, 20):
    print(f"{L:6}  {observed[L] / len(draws):13.4f}  {expected[L]:17.4f}")

Output

length  sampled share  model probability
    10         0.0050             0.0048
    11         0.0303             0.0308
    12         0.0769             0.0768
    13         0.1704             0.1712
    14         0.2264             0.2254
    15         0.2370             0.2370
    16         0.1373             0.1374
    17         0.0649             0.0650
    18         0.0290             0.0289
    19         0.0126             0.0124

Full program, including the set-up hidden here

Lengths of 50,000 sequences drawn from the reference model, against its length distribution computed from the model.

Sampling in motion

Loading…

Press Draw to watch sequences being drawn from the reference model, newest first, with their log probabilities. The bars show the share of each length among the draws so far; the dots are the model's length distribution, computed rather than sampled, which the bars approach as draws accumulate.

Not every draw is a textbook CDR3

The reference model gives some probability to every sequence of up to 27 residues, including sequences it has never seen. Most of what it draws has the conserved start and end, but not all:

canonical.py

import BEAM

model = BEAM.load_pretrained("human_tcrb_cdr3_aa_foundation")
draws = model.sample(50_000, seed=11)

starts_c = sum(s.startswith("C") for s in draws) / len(draws)
ends_fw = sum(s.endswith(("F", "W")) for s in draws) / len(draws)
print(f"start with C:      {starts_c:.1%}")
print(f"end with F or W:   {ends_fw:.1%}")

Output

start with C:      98.0%
end with F or W:   96.9%

Full program, including the set-up hidden here

That is a little below the corpus the reference model was built from, in which, according to the paper’s Methods, 99.6% of sequences start with C and 98.6% end with F or W. The model reserves some probability for sequences it never observed, and some of that goes to non-canonical ones. Filter the draws if your use needs canonical CDR3s only.

Pitfalls

• Draws reproduce what the model was built from. Each draw is one cell drawn independently from the population the model describes, so repeats come only from probability. A model built from one person’s repertoire, weighted by abundance, gives expanded clones high probability, and its draws repeat them at roughly their frequency in that repertoire. The reference model pools many people, so its draws reflect no one person’s clones; a model of unique sequences has no clonal structure to reproduce. See Abundance or unique sequences?
• A fixed length changes the question. sample(n, length=12) draws from the sequences of length 12 only. Use it when length is controlled by design, not to imitate a repertoire.