Research notes
Plain-language definitions, visual intuitions in words, and the assumptions that matter. These terms come from the research articles, with links to their examples and sources.
Share stored data until a writer needs its own changed copy.
Copy-on-write lets sequences share KV-cache blocks while their contents remain identical. A sequence that must modify a shared block receives a private copy, preserving the others’ state.
An intuition: share one document until someone needs a different revision.
Read in context: How much KV-cache space do variable-length requests waste? · Source or context
The timer starts when the work is credited, so any line before service counts too.
Effective latency runs from credit issuance through response completion. Comparing it with request latency shows how much time passed before dispatch.
Read in context: Where LLM inference benchmarks hide queueing delay · Source or context
Like the line at a store: the request that arrives first is served first.
A first-in, first-out queue runs requests in their arrival order. Here, only one request uses the service slot at a time.
Read in context: Where LLM inference benchmarks hide queueing delay · Source or context
Several query heads share each set of key and value heads.
Grouped-query attention divides query heads into groups that share key/value heads. It can use fewer KV heads than query heads, reducing cached state relative to full multi-head attention with otherwise matching dimensions.
An intuition: several questioners consult the same set of notes. Count KV heads in the cache-size formula, not query heads; quality and speed still depend on the model and implementation.
Read in context: The KV cache is the memory wall · Source or context
Reconsider the active request batch after each generation iteration.
Iteration-level scheduling asks the engine to execute one generation iteration at a time, allowing finished requests to leave and waiting ones to enter at iteration boundaries.
An intuition: refill a seat as soon as it becomes free instead of waiting for the whole row to empty.
Read in context: Why should an LLM server reschedule after every token? · Source or context
Saved attention keys and values from earlier tokens, reused when generating the next token.
A KV cache stores earlier tokens’ key and value tensors for reuse during autoregressive decoding. A simple full-cache budget is 2 × layers × live tokens × KV heads × head dimension × bytes per element, with batch accounted for in live tokens.
An intuition: keep reusable notes from earlier steps. The cache avoids recomputing old projections, but attending to a growing history still takes work.
Read in context: The KV cache is the memory wall · Source or context
An attention method that reads cached keys and values from separately allocated blocks.
PagedAttention allows a sequence’s KV state to occupy noncontiguous fixed-size blocks, using mappings between logical token blocks and physical storage. Allocating blocks as needed reduces wasted reservations.
An intuition: a book’s pages can live in separate drawers if an index tells you where each page is.
Read in context: How much KV-cache space do variable-length requests waste? · Source or context
The phase that processes the prompt and builds attention state for generation.
Prefill processes a request’s input tokens and builds the key-value state used by later decoding. The first output token may be emitted at its end, depending on the serving design.
An intuition: read the supplied context before continuing the text.
Read in context: When is it worth separating prompt processing from token generation? · Source or context
Batch selected model operations while handling others differently.
Orca’s selective batching applies cross-request batching to selected operations, enabling requests at different token positions to share a generation iteration. It does not mean every operation uses one uniform batch representation.
An intuition: combine the assembly steps that fit together while keeping incompatible steps separate.
Read in context: Why should an LLM server reschedule after every token? · Source or context
A way to describe the slower end of many requests, rather than the typical request.
A percentile such as p95 is a rank-based summary: roughly 95% of observations are at or below the reported value under the selected convention. It is not the maximum.
Read in context: Where LLM inference benchmarks hide queueing delay · Source or context
A decoding-speed measure whose averaging window must be stated.
Time per output token summarizes decoding time per generated token, commonly as a per-request average after the first token. A good average can hide one long gap, so it is not a worst-gap guarantee.
An intuition: a smooth average pace can include a noticeable pause.
Read in context: When is it worth separating prompt processing from token generation? · Source or context
Combine values from every worker and give the combined result back to every worker.
An all-reduce applies a reduction, such as addition, across participating workers and returns the result to each one.
An intuition: everyone contributes a subtotal and receives the same grand total. Tensor-parallel layers use it to combine partial outputs. Communication volume depends on the collective algorithm; floating-point results can depend on reduction order.
Read in context: Tensor Parallelism’s Communication Boundary · Source or context
The saved state needed to pick up a process where it left off.
A training checkpoint may need model parameters, optimizer buffers, random-number-generator state, and other execution state. Saving only parameters may not be enough for exact continuation.
Read in context: 100 identical loss logs, 100 different model states · Source or context
Rerunning the same steps to check whether they lead to the same state.
A replay check fixes relevant inputs and execution conditions, then compares a rerun with a reference. Exact bits are stricter than similar loss or accuracy.
Read in context: 100 identical loss logs, 100 different model states · Source or context
Let jobs take turns on a GPU rather than finish one job before starting the next.
GPU time-slicing suspends and resumes jobs so they share accelerator time. It can expose early feedback sooner while adding switching overhead and delaying some completions.
An intuition: give each experiment an initial turn before spending the whole afternoon on the first.
Read in context: When does GPU time-slicing speed up useful feedback? · Source or context
A smaller piece of a training batch that can move through pipeline stages.
A microbatch is one subdivision of a larger training batch. GPipe sends these pieces through layer partitions and accumulates their gradients before a synchronous update.
An intuition: smaller parcels keep an assembly line occupied instead of moving one oversized parcel at a time.
Read in context: How do microbatches change pipeline idle time? · Source or context
Idle time while a pipeline fills, drains, or waits for dependencies.
A pipeline bubble is unused stage time caused by the schedule’s dependencies. More microbatches can reduce fill-and-drain overhead relative to useful work, but an unevenly slow stage still limits progress.
An intuition: a station sits empty while it waits for the next parcel.
Read in context: How do microbatches change pipeline idle time? · Source or context
Adding the same numbers in a different order can change the last few digits.
Floating-point addition rounds intermediate results. Reordering a sum can therefore change low-order bits even when the same values are included.
Read in context: 100 identical loss logs, 100 different model states · Read in context: Why parallel sums disagree: tracing FP32 reduction order · Source or context
How much learning progress is obtained from the examples processed.
Statistical efficiency describes how effectively training examples contribute to progress. Larger batches can reduce gradient noise but need not provide proportionally more progress per update. Pollux models this alongside throughput.
An intuition: repeating a known example may consume effort without adding much information.
Read in context: Why can more training examples per second mean slower learning? · Source or context
Split the computations inside a model layer across devices.
Tensor parallelism partitions a layer’s tensors and operations across workers. Some intermediate results stay local; others require communication before the next operation.
An intuition: several workers calculate different parts of one large worksheet, then combine the parts that the next step needs. The placement of nonlinear operations helps determine where communication is necessary.
Read in context: Tensor Parallelism’s Communication Boundary · Source or context
Training throughput adjusted for how much useful learning the processed examples provide.
Pollux combines system throughput with statistical efficiency to estimate useful training progress per unit time. This differs from serving goodput defined by latency-qualified requests.
An intuition: count how much material is learned per hour, not merely how many pages are turned.
Read in context: Why can more training examples per second mean slower learning? · Source or context
Divide training state across workers so each worker does not keep every copy.
ZeRO progressively partitions optimizer state, gradients, and parameters across data-parallel workers. Required pieces are communicated when needed.
An intuition: each worker owns part of a shared filing cabinet instead of a complete duplicate. Persistent-state savings do not include every activation, gathered parameter, or temporary buffer that contributes to peak memory.
Read in context: ZeRO’s Memory Budget, Byte by Byte · Source or context
A small menu of allowed values; each input uses the closest option.
A quantizer assigns each input coordinate to a codebook entry and stores its index. The number and placement of entries affect storage and reconstruction error.
Read in context: Three-bit vectors, three different retrieval outcomes · Source or context
A requirement every acceptable candidate must satisfy.
A hard constraint is a condition whose violation disqualifies a candidate, unlike a preference that can trade off against other benefits. The WebShop-inspired fixture requires color, size, and price conditions together; relevance alone does not establish feasibility.
An intuition: a well-ranked jacket is still unsuitable if it is outside the allowed budget.
Read in context: Retrieval relevance is not constraint satisfaction · Source or context
Find the stored vector with the largest dot product with a query vector.
Maximum inner product search ranks database vectors by their dot product with a query.
An intuition: ask which stored arrow projects most strongly onto the query’s direction, accounting for vector lengths. It is not automatically cosine similarity or nearest Euclidean distance; those relationships require conditions such as normalization.
Read in context: The cost of asking a learned index for a key · Source or context
A ranking score that rewards relevant results near the top, relative to the best possible ordering.
nDCG@k discounts graded relevance by rank within the top k results, then normalizes by an ideal ordering under the same gain rule and candidate set.
An intuition: valuable results earn more credit when the reader sees them sooner. The labels, gain formula, candidate pool, and zero-ideal convention are part of the metric; unjudged items are not automatically known negatives.
Read in context: Every query pays the same price, and that is the bug · Source or context
Of the best matches found with the full data, how many are still found after compression?
Recall@k compares approximate neighbors with an exact reference set of k neighbors. It measures overlap, not reconstruction error or latency.
Read in context: Three-bit vectors, three different retrieval outcomes · Source or context
For a direction, report the largest dot product achieved by any vector in a set.
A set’s support function maps a query direction to the maximum dot product over that set. For a finite database, wherever the function is differentiable, its gradient is the maximizing key. Ties between distinct maximizing keys can make the function nondifferentiable.
An intuition: slide a straight boundary toward a cloud of points and ask which point it touches last in the chosen direction.
Read in context: The cost of asking a learned index for a key · Source or context
A compact code that stands in for a longer list of numbers.
A quantizer maps vector coordinates or subvectors to a finite set of representative values. Fewer bits reduce storage, while reconstruction and retrieval quality can change.
Read in context: Three-bit vectors, three different retrieval outcomes · Source or context
A reference method or result used to judge what a proposed change adds.
A baseline provides a comparison under stated data, metrics, and resource conditions. It can be simple or sophisticated, but its setup must be clear.
An intuition: keep a reference ruler beside the new result. Improving a score means little if the ruler changed too, such as using a different test set, candidate pool, or compute budget.
Read in context: How I got into the OpenAI Student Collective · Read in context: Every query pays the same price, and that is the bug · Source or context
An aggregate that computes each group’s score and then weights groups equally.
A macro average first computes a score within each family, then averages those family scores equally. Small families therefore matter as much as large ones. The article uses this illustrative rule rather than AgentBench’s official normalization.
An intuition: give each school subject one equal vote, regardless of how many questions it contains.
Read in context: When benchmark averages reverse an agent ranking · Source or context
An aggregate that pools individual task outcomes, giving each task equal weight.
A micro average pools outcomes across tasks before computing the overall rate, so larger task families contribute more. The article contrasts this with equal-family weighting; neither is AgentBench’s official normalized score.
An intuition: if eighty of a hundred questions concern lookup, lookup largely determines the final mark.
Read in context: When benchmark averages reverse an agent ranking · Source or context
A benchmark’s rules for how inference requests arrive and how performance is assessed.
MLPerf scenarios specify a load pattern and performance requirements. Offline supplies a batch of samples; Server sends individual queries with Poisson arrivals and latency constraints.
An intuition: compare processing a full tray with serving people who arrive over time. Their throughput ratio depends on workload and implementation; it is not a universal conversion or utilization percentage.
Read in context: The Server/Offline ratio is workload-specific · Source or context
A simulated evaluation that tests the workflow using scripted or sampled stand-ins for real model behavior.
A mock evaluation substitutes configured outputs or probability rules for actual model and grader calls. It can test data flow and interfaces.
An intuition: a rehearsal checks that the microphones work; it does not measure the eventual performance. More mock trials estimate the chosen simulation more precisely, not the real model’s quality.
Read in context: Audit the evaluation before trusting the score · Source or context
A cost assigned by assumptions or a formula, rather than measured from a running system.
A cost model estimates expense using declared inputs, such as fixed milliseconds per route. It can compare choices under those assumptions.
An intuition: a travel budget is useful before a trip, but it is not a stopwatch record. Modeled savings become latency claims only after measuring the relevant system, workload, and timing boundary.
Read in context: Every query pays the same price, and that is the bug · Read in context: The cost of asking a learned index for a key · Source or context
The chance of finding at least one successful attempt among k tries at the same task.
Pass@k measures whether at least one of k independent, identically distributed attempts succeeds, then averages across tasks.
An intuition: several keys are tried, and opening the door once is enough. For one task with success probability p it is 1 − (1 − p)^k. Shared failures or changing conditions can invalidate the independent-trial interpretation.
Read in context: Reliability Beyond One Successful Run · Source or context
The chance that every one of k attempts at the same task succeeds.
Pass^k measures all-k success under independent, identically distributed trials, averaged across tasks.
An intuition: a door must open on every visit, not just once. For a task with success probability p, all-k success is p^k. Average those task-level values; raising the overall average to k generally gives a different answer. Observed finite samples use a subset estimator.
Read in context: Reliability Beyond One Successful Run · Source or context
Checking whether the resulting environment satisfies the requested outcome.
State-based evaluation judges a task through relevant environment state rather than merely trusting an agent’s report. WebArena uses task-dependent evidence and checks; the article’s cart predicate is a small example.
An intuition: inspect the actual cart contents instead of accepting an “added successfully” banner.
Read in context: WebArena’s evaluation lesson: check the final state · Read in context: The gap between a fresh screenshot and a safe tap · Source or context
A measure of how differently two probability distributions divide their weight among outcomes.
For discrete distributions, total variation distance is half the sum of their absolute probability differences.
An intuition: it is the amount of probability mass that must move to turn one mixture into the other. When fixed subgroup scores lie between zero and one, it bounds the aggregate-score change caused only by changing subgroup weights.
Read in context: The denominator inside a geospatial benchmark · Source or context
Connecting a requested object or operation to the correct actionable element.
Action grounding maps a task’s intended target to a current element identifier, region, or coordinate that an interface can act on. Recognizing the desired object and identifying its control are separate requirements.
An intuition: spotting the red mug is not enough if the click goes to the blue mug’s button.
Read in context: What text-only observations erase in visual web tasks · Read in context: Why normalized clicks miss fixed-margin controls · Source or context
Positions expressed as fractions of an image or viewport’s width and height.
Normalized coordinates express a point relative to the containing width and height. Reusing those fractions after resizing assumes proportional layout changes. This is the article’s geometric baseline, not an OSWorld guarantee.
An intuition: “three quarters across” follows a stretching canvas, but not a button fixed fifty pixels from its right edge.
Read in context: Why normalized clicks miss fixed-margin controls · Source or context
Whether the evidence an agent uses still describes the world when it acts.
Observation freshness concerns whether an observation remains applicable when a decision or action uses it. Refreshing reduces stale evidence but does not prevent a later change before execution. This is the local ReAct-inspired experiment’s distinction.
An intuition: checking a shelf now is better than remembering yesterday’s stock, but someone can still take the item before you reach it.
Read in context: When should a ReAct-style agent refresh its observations? · Read in context: The gap between a fresh screenshot and a safe tap · Source or context
The specification of which environment state must be restored before a trial.
A reset contract identifies all relevant state that must return to its specified starting condition. BrowserGym distinguishes environment interaction from backend preparation; the article tests cookies, counters, and a cache.
An intuition: restarting a board game requires resetting the pieces and score, not merely rolling with the same random seed.
Read in context: A fixed seed is not a reset contract · Source or context
A remembered lesson is used only under the conditions in which it applies.
Scoped memory attaches applicability conditions, such as a rule version, to stored advice and retrieves it only when those conditions hold. The article proposes this extension; Reflexion itself uses bounded textual feedback across trials.
An intuition: a note saying a road is closed should apply during the closure, not ban that road forever.
Read in context: Memory needs a scope: a rule-change test inspired by Reflexion · Source or context
Removing branches from consideration to reduce search work.
Search pruning discards candidate branches according to a rule or score. It saves evaluations but may eliminate the best continuation when its early score is misleading. The article’s permanently pruned branches cannot return just because the budget grows.
An intuition: crossing a route off the map saves exploration, but extra time cannot help if you never reconsider it.
Read in context: What a search budget cannot fix after early pruning · Source or context
Error details organized into fields that help a caller choose a repair.
Structured feedback reports actionable details such as an error category, location, and valid range. An agent interface can make recovery easier by exposing these distinctions. The local article uses scripted repairs, not measured language-model improvement.
An intuition: “line 12 exceeds this two-line file” tells you more than “edit failed.”
Read in context: Structured edit errors make recovery programmable · Source or context
Keeping candidate tool calls whose results meet a usefulness criterion.
Toolformer filters sampled API calls using their effect on prediction loss for subsequent text, then trains on retained calls. This training criterion differs from the article’s invented runtime error-and-latency cost.
An intuition: keep worked examples where consulting a tool actually helps predict what comes next.
Read in context: Selective tool calls: finding the cost threshold · Source or context
Exploring branching sequences of possible decisions to find a useful outcome.
Tree search represents alternative decisions as branches and allocates evaluations to possible continuations. LATS combines this with language-model actions, evaluations, feedback, and reflection. The local nine-leaf fixture tests only search allocation.
An intuition: examine several routes through a maze instead of committing to the first promising turn.
Read in context: What a search budget cannot fix after early pruning · Source or context
A boundary that keeps external content from becoming permission to act.
A trust boundary separates information from the authority needed to execute an operation. The article checks proposals against an independently supplied grant; InjecAgent studies attacks carried in tool responses, not this validator’s effectiveness.
An intuition: a document can ask you to archive it, but the request written inside it is not the user’s permission.
Read in context: A tool response cannot grant execution authority · Source or context
A sequential decision problem with reward and explicit expected-cost limits.
A constrained Markov decision process optimizes expected return over policies subject to auxiliary expected-cost budgets. The cost definition, discounting, and starting distribution are part of the specification.
An intuition: maximize performance while keeping a separately defined accounting total within budget.
Read in context: Is a safety constraint the same as a large negative reward? · Source or context
An imitation-learning method that asks the expert about states reached during rollouts.
DAgger collects states under learner or expert-learner rollout policies, asks an expert for actions at those states, aggregates the labeled data, and retrains.
An intuition: practice recovery from the places your own mistakes take you, rather than only copy a flawless demonstration.
Read in context: Why does an imitation learner struggle after its first mistake? · Source or context
Each agent chooses actions from its own available information at execution time.
Decentralized execution lets each agent act using its own observations and history rather than requiring global state at execution time. Centralized training may still use extra information, as in QMIX.
An intuition: a coach can study the full field during practice, but each player must choose using what that player can observe during play.
Read in context: The payoff tables a monotonic QMIX mixer cannot represent · Source or context
The coefficient that trades expected reward against policy entropy.
The entropy temperature sets the weight of entropy relative to reward. In the finite-action example, higher positive temperature spreads probability more evenly; scaling reward without scaling temperature changes that tradeoff.
An intuition: it is the price assigned to keeping options open.
Read in context: Why would a controller be rewarded for staying unpredictable? · Source or context
A bound on average policy cost, not a guarantee for every trajectory.
An expected-cost constraint limits cost averaged over the policy and environment’s possible trajectories, often with discounting. Individual trajectories can exceed the budget even when the expectation satisfies it.
An intuition: an average spending cap does not limit each individual purchase.
Read in context: Is a safety constraint the same as a large negative reward? · Source or context
Store past transitions and sample them again for learning updates.
Experience replay reuses stored transitions instead of training only on the newest interaction sequence. Random sampling changes training order but cannot create unobserved states or guarantee independent data.
An intuition: shuffle a notebook of past examples rather than repeatedly studying only its latest page.
Read in context: Why does an agent learn from old experiences? · Source or context
Learning a policy from examples of another policy’s actions.
Imitation learning trains a policy using expert behavior. In sequential tasks, its own actions change later observations, so demonstration data may omit states the learner reaches.
An intuition: watching an expert stay on course may not teach how to recover after a wrong turn.
Read in context: Why does an imitation learner struggle after its first mistake? · Source or context
Learning about a policy using experience collected under another policy.
Off-policy learning uses data whose behavior policy may differ from the policy being evaluated or improved. Old replay transitions are useful only with an update rule that supports this mismatch.
An intuition: learn which choices are valuable from a log collected by an earlier decision maker.
Read in context: Why does an agent learn from old experiences? · Source or context
A measure of how spread out a policy’s action probabilities are.
Policy entropy measures uncertainty in the action distribution at a state. Maximum-entropy RL adds a weighted entropy term to reward, changing the objective rather than just perturbing a chosen action.
An intuition: the objective pays something for keeping several choices in play.
Read in context: Why would a controller be rewarded for staying unpredictable? · Source or context
A reward bonus built from discounted differences of state potentials.
Potential-based shaping adds the discounted next-state potential minus the current-state potential. With the required boundary conditions, discounted bonuses cancel to a policy-independent offset, preserving optimal policies.
An intuition: move credit along a route without changing which complete route earns more.
Read in context: When does a reward bonus change the task an agent learns? · Source or context
A training-objective rule that stops rewarding some overly large policy changes.
PPO’s clipped surrogate takes the smaller of an advantage-weighted probability ratio and its clipped version. The advantage’s sign determines which direction loses incentive.
An intuition: the reward for pushing farther can flatten without a wall preventing movement. Other samples and shared parameters can still move the ratio outside the clip range; clipping is not a hard bound on policy change.
Read in context: What PPO Clipping Actually Limits · Source or context
Raising one agent utility cannot lower the mixed team value when other inputs are fixed.
QMIX constrains its mixer to be nondecreasing in each agent utility. With independently available actions, local utility maximizers therefore form a joint maximizer of the mixed value. This is sufficient, not necessary, for decentralized optimal choices.
An intuition: turning one utility dial upward cannot make the team’s displayed value go down.
Read in context: The payoff tables a monotonic QMIX mixer cannot represent · Source or context
The potential assigned to the true final state in a shaped-return calculation.
Terminal potential is the shaping function’s value at a terminal state. Setting it to zero removes the trajectory-dependent endpoint term in the article’s finite episodic calculation. A truncated rollout is not automatically a true terminal.
An intuition: closing a partial ledger too early can leave an uncancelled balance.
Read in context: When does a reward bonus change the task an agent learns? · Source or context
The amount of data needed in flight to keep a path busy during its baseline round trip.
For the path model here, bandwidth–delay product is bottleneck bandwidth multiplied by round-trip propagation time. Use matching units, such as bytes per second times seconds.
An intuition: enough data must remain on the road while acknowledgments return. Beyond that amount, extra inflight data can build a queue instead of increasing steady throughput.
Read in context: The Bandwidth–Delay Product Knee · Source or context
A sender-side limit on how much unacknowledged data can be in flight.
A TCP congestion window limits outstanding data as part of congestion control. It is distinct from physical queue capacity and does not alone determine throughput.
An intuition: control how many parcels are on the road awaiting confirmation, rather than how many fit in one warehouse.
Read in context: How can congestion feedback protect short requests from large transfers? · Source or context
The share of packets carrying congestion marks in an observation window.
DCTCP estimates congestion extent from the fraction of marked packets, smoothing successive observations before reducing its congestion window. The fraction conveys more than whether any mark occurred.
An intuition: one warning among a hundred messages suggests a different situation from a hundred warnings.
Read in context: How can congestion feedback protect short requests from large transfers? · Source or context
The time a signal spends traveling along a path.
Propagation delay is path length divided by signal speed in the transmission medium, summed across path segments.
An intuition: a longer journey can finish sooner if its route allows faster travel. It excludes queueing, processing, and serialization delays, so a modeled propagation advantage is not a measured application-latency guarantee.
Read in context: When a Longer Space Route Is Faster · Source or context
An assessment of viewing experience that can combine quality, stalls, and quality changes.
A streaming QoE objective combines chosen measures such as visual quality, stall time, and quality variation. Its weights express tradeoffs and are not universal measurements of viewer preferences.
An intuition: a sharper picture may be less enjoyable if it repeatedly freezes.
Read in context: Will a learned streaming policy handle a network it has never seen? · Source or context
A playback pause caused by running out of buffered video.
Rebuffering occurs when playback consumes the available video before the next chunk arrives. In the article’s simplified model, stall duration is download time beyond the buffer available at download start.
An intuition: the screen pauses when the reserve of ready-to-play seconds runs out.
Read in context: Will a learned streaming policy handle a network it has never seen? · Source or context
The time a packet spends in a queue before it leaves.
Packet sojourn time measures elapsed time between entering and leaving a queue. CoDel uses it to distinguish persistent queueing delay from short bursts.
An intuition: time spent waiting tells you more about the line than its length alone.
Read in context: Why can a bigger network buffer make an application slower? · Source or context
A backlog that persists instead of draining between bursts.
A standing queue is persistent queued work that adds waiting time even when brief bursts have passed. CoDel checks whether delay stays high over an interval, with protection for nearly empty queues.
An intuition: a line that never clears makes every new arrival wait.
Read in context: Why can a bigger network buffer make an application slower? · Source or context
A compact membership check that can say “definitely absent” or “possibly present.”
A standard Bloom filter uses hashed bit positions to screen membership. With correct construction and use, it has no false negatives, but a positive answer may still be false.
An intuition: a cheap checkpoint rules out many unnecessary file searches, while uncertain cases continue to the real data. More bits per key usually reduce false positives under the chosen hashing model.
Read in context: Giving Small Runs Better Bloom Filters · Source or context
Concurrent operations behave as if each took effect at one instant between its start and finish.
Linearizability requires a legal sequential ordering of operations that respects real-time order for non-overlapping calls.
An intuition: concurrent requests can be placed on one consistent timeline, with each effect occurring during its call. For registers this supports an atomic read/write abstraction. Counting overlapping quorums is only one ingredient in a protocol that provides it.
Read in context: What Quorum Arithmetic Leaves Out · Source or context
A required set or number of replica responses used by a read or write protocol.
For a fixed set of N replicas, read and write response sets of sizes R and W must intersect if R + W > N.
An intuition: two sufficiently large groups drawn from the same room must share someone. Overlap alone does not establish linearizability: membership, durable state, version selection, and the read/write protocol still matter.
Read in context: What Quorum Arithmetic Leaves Out · Source or context
Arrange neighboring GPU threads’ global-memory accesses so hardware can service them with fewer transactions.
Coalescing combines a warp’s global-memory accesses into the transactions needed for their addresses. Contiguous, suitably aligned accesses often need fewer transactions than widely spaced ones.
An intuition: collect nearby parcels in a small number of trips. Exact transaction counts depend on access size, alignment, and GPU architecture.
Read in context: CUDA Transpose: Coalescing and Bank Padding · Source or context
Compute normalized attention weights in pieces, keeping a running maximum and rescaled totals.
Online softmax updates a running maximum and exponential sum as score blocks arrive. Earlier totals are rescaled when the maximum changes; attention can also maintain a weighted-value sum.
An intuition: keep a running subtotal in consistent units instead of retaining every receipt. It reorganizes exact attention arithmetic, subject to floating-point rounding, rather than dropping token pairs.
Read in context: Online Softmax Without the Attention Matrix · Source or context
GPU threads request different addresses in the same shared-memory bank, forcing extra servicing steps.
Shared memory is divided into banks. When a warp requests distinct addresses that collide in a bank, those accesses may be serialized; supported broadcast cases differ.
An intuition: several requests queue at one service counter while others are free. Padding a tile can change address-to-bank mapping, but the useful padding depends on layout, element size, and hardware.
Read in context: CUDA Transpose: Coalescing and Bank Padding · Source or context
A bit mask identifying the GPU threads that must join a warp-level collective.
A warp participation mask specifies which lanes participate in a synchronized CUDA collective. The participating threads must obey the operation’s synchronization and data-exchange contract; a mask does not create missing input values.
An intuition: a roll call identifies who must join the group calculation, but does not supply an absent person’s number.
Read in context: Why parallel sums disagree: tracing FP32 reduction order · Source or context
Variation in outcomes that remains even when the modeled conditions are fixed.
Aleatoric uncertainty represents outcome variability conditional on the information supplied to a model, such as noise in observed state transitions.
An intuition: even a well-characterized process can produce a spread of outcomes. A probabilistic model estimates that spread; more training data can improve the estimate without eliminating the underlying variability.
Read in context: When Dynamics Models Agree and Still Fail · Source or context
A top-down representation that places observations into a shared map-like coordinate frame.
A bird’s-eye-view representation organizes features by position in a common top-down grid. Image pixels alone give viewing directions, so depth and camera geometry are needed to place them there.
An intuition: turn views from several cameras into marks on the same floor plan. Uncertain depth, calibration errors, and grid size each affect where those marks land.
Read in context: From Camera Pixels to a Bird’s-Eye Grid · Source or context
A function whose conditions help a controller keep the system inside an allowed set of states.
A control barrier function provides an inequality on the dynamics and control that can establish forward invariance of a safe set under stated assumptions.
An intuition: a margin warns the controller before it crosses a boundary. The guarantee depends on model accuracy, regularity, feasible controls, and how commands are applied; a sampled or saturated implementation needs its own analysis.
Read in context: When a Barrier Filter’s Assumptions Break · Source or context
Uncertainty caused by incomplete knowledge of the system being predicted.
Epistemic uncertainty concerns what a model has not learned or cannot identify from available evidence. An ensemble’s disagreement can be a useful proxy.
An intuition: several maps disagree because an area has not been surveyed well. Agreement is not proof of accuracy: all models can share the same missing information or bias.
Read in context: When Dynamics Models Agree and Still Fail · Source or context
Place image features along possible depths, then accumulate them into a shared spatial grid.
In Lift, Splat, Shoot, lifting assigns image features to depth hypotheses along camera rays; splatting pools the transformed features into bird’s-eye grid cells.
An intuition: spread one pixel’s evidence over plausible places, then add evidence that lands in the same cell. The weighted accumulation does not pick one certain depth or remove calibration error.
Read in context: From Camera Pixels to a Bird’s-Eye Grid · Source or context
Tests that fail before a repair and must pass afterward.
FAIL_TO_PASS is SWE-bench’s partition of tests expected to change from failing to passing after a successful repair. Resolution also requires the PASS_TO_PASS tests to remain passing; missing required results fail evaluation.
An intuition: first check that the repaired door now opens, then separately check that its lock still works.
Read in context: A patch needs two test partitions · Source or context
Exercise software with many generated or mutated inputs to find crashes or broken properties.
Fuzz testing repeatedly supplies varied inputs and checks for failures such as crashes, invalid memory access, or violated assertions.
An intuition: try many unusual shapes of key, not only the ones in a hand-written test list. A useful check must reflect the API’s actual contract; a false assertion can flag valid behavior as a bug.
Read in context: How to contribute to open source at big tech · Source or context
Previously passing tests that must still pass after a change.
PASS_TO_PASS is SWE-bench’s partition checking that previously passing behavior remains passing after a repair. These regression checks complement tests of the reported bug; passing the selected suite does not cover every possible behavior.
An intuition: fixing the reverse gear should not break the forward gears.
Read in context: A patch needs two test partitions · Source or context
Match entries by where they appear in an ordered sequence.
Positional matching uses an entry’s place in an ordered sequence to select its counterpart. The stage-profile loader enables this only under its compatibility rules; other cases use names or occurrence labels.
An intuition: numbered seats distinguish two people with the same name. Reordering or omitting entries can change the meaning, so the matching contract must be explicit.
Read in context: How I got into the MLH Fellowship · Source or context
An edit that commits only after its required checks pass, otherwise preserving the original.
A transactional edit prepares a candidate change, validates it, and replaces the original only if the required checks pass. SWE-agent’s guarded editor motivates the local parser-based example. Acceptance proves only the checks that ran, not full program correctness.
An intuition: review a replacement page before swapping it into the binder.
Read in context: Structured edit errors make recovery programmable · Source or context
The longest duration-weighted dependency chain in a job.
A critical path is a dependency chain with the greatest total duration. Its length lower-bounds completion time even with unlimited workers, while limited workers and other constraints can add delay.
An intuition: extra cooks cannot make a later step start before its ingredients are ready.
Read in context: What does a scheduler lose when it ignores job dependencies? · Source or context
One request launches work on several downstream workers.
Fan-out sends parts of a request to multiple workers. If completion requires every result, the slowest worker determines completion time.
An intuition: a group trip cannot depart until the last required passenger arrives.
Read in context: Why does one slow server matter more as a service grows? · Source or context
Sharing work across different kinds of devices that do not all run it at the same speed.
A heterogeneous scheduler accounts for how each job progresses on each device type, along with allocation and sharing constraints.
Read in context: How should AI jobs share different GPUs? · Source or context
A model assumption that one worker’s latency tells you nothing about another’s.
Independent worker times allow their deadline-success probabilities to multiply. If they also share one distribution, all-worker success is its single-worker probability raised to the worker count. Shared congestion can break independence.
An intuition: separate coin tosses differ from several clocks stopped by the same power outage.
Read in context: Why does one slow server matter more as a service grows? · Source or context
A job’s arrival-to-completion time divided by its processing duration.
Job slowdown normalizes completion time by the job’s own processing duration. An equal wait therefore penalizes a short job more than a long one.
An intuition: waiting ten minutes is a larger interruption for a one-minute task than for an all-day task.
Read in context: Can a scheduler learn to pack two resources at once? · Source or context
Schedule jobs subject to several resource capacities at once.
Multi-resource scheduling accounts for each job’s simultaneous demands, such as CPU and memory. A feasible placement must respect every capacity; tight packing alone does not optimize completion time or slowdown.
An intuition: a suitcase must satisfy both its volume limit and the airline’s weight limit.
Read in context: Can a scheduler learn to pack two resources at once? · Source or context
How quickly a system makes progress on work.
Throughput is a rate over time. In a scheduler model, effective throughput describes expected progress under allocated time shares and measured device rates.
Read in context: How should AI jobs share different GPUs? · Source or context