Why normalized clicks miss fixed-margin controls notebook

Python starts when you choose Run.

Why normalized clicks miss fixed-margin controls

A viewport-relative point and a right-anchored control obey different transformations. Target width decides when that difference becomes a miss.

Pure geometry stress test; element positions are oracle inputs, not tested visual grounding.

"""Why normalized clicks miss fixed-margin controls
Deterministic, standard-library experiment. RESULTS contains the full data.
"""
from math import hypot
VIEWPORTS = [(800, 600), (1200, 900), (1000, 700), (600, 800), (1400, 1000)]
layouts = [(kind, i) for kind in ('proportional', 'right_margin') for i in range(5)]

def center(layout, viewport):
    kind, i = layout
    width, height = viewport
    x = width * (0.2 + 0.1 * i) if kind == 'proportional' else width - 50 - 5 * i
    return x, height / 2

for kind in ('proportional', 'right_margin'):
    for policy in ('pixel', 'normalized', 'element'):
        hits, errors = 0, []
        for layout in [x for x in layouts if x[0] == kind]:
            old = center(layout, VIEWPORTS[0])
            for viewport in VIEWPORTS:
                target = center(layout, viewport)
                if policy == 'pixel':
                    click = old
                elif policy == 'normalized':
                    click = old[0] * viewport[0] / 800, old[1] * viewport[1] / 600
                else:
                    click = target  # Oracle current element center, not learned grounding.
                dx, dy = click[0] - target[0], click[1] - target[1]
                hits += abs(dx) <= 10 and abs(dy) <= 10
                errors.append(hypot(dx, dy))
        print(kind, policy, f'hits={hits}/25', f'mean_error={sum(errors)/25:.1f}px')
assert center(('right_margin', 0), (1200, 900))[0] == 1150

rows = []
for layout in layouts:
    for width,height in VIEWPORTS:
        target=center(layout,(width,height)); old=center(layout,VIEWPORTS[0])
        for policy in ('pixel','normalized','element'):
            click=old if policy=='pixel' else ((old[0]*width/800,old[1]*height/600) if policy=='normalized' else target)
            dx,dy=click[0]-target[0],click[1]-target[1]
            rows.append(dict(kind=layout[0],layout=layout[1],width=width,height=height,policy=policy,target=list(target),click=list(click),dx=dx,dy=dy,error=hypot(dx,dy),hit=abs(dx)<=10 and abs(dy)<=10))
tolerance=[]
for half_width in (5,10,20,40,80):
    tolerance.append(dict(half_width=half_width,hits=sum(abs(r['dx'])<=half_width and abs(r['dy'])<=half_width for r in rows if r['kind']=='right_margin' and r['policy']=='normalized')))
RESULTS=dict(cases=rows,target_tolerance=tolerance)
print('Right-margin normalized hit counts by half-width:',tolerance)

proportional pixel hits=5/25 mean_error=153.2px
proportional normalized hits=25/25 mean_error=0.0px
proportional element hits=25/25 mean_error=0.0px
right_margin pixel hits=5/25 mean_error=297.9px
right_margin normalized hits=5/25 mean_error=21.0px
right_margin element hits=25/25 mean_error=0.0px
Right-margin normalized hit counts by half-width: [{'half_width': 5, 'hits': 5}, {'half_width': 10, 'hits': 5}, {'half_width': 20, 'hits': 15}, {'half_width': 40, 'hits': 21}, {'half_width': 80, 'hits': 25}]

Python runs locally in this page. Saved results and figures are visible before execution.

Download full experiment notebook