{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# From Camera Pixels to a Bird’s-Eye Grid\n",
        "\n",
        "Depth hypotheses place a pixel along several points on a camera ray. Their weighted projection exposes the effects of calibration and grid resolution.\n",
        "\n",
        "Pure coordinate geometry and weighted grid accumulation for one synthetic pixel; no camera, neural network, training, or perception benchmark.\n",
        "\n",
        "The code uses Python 3 and the standard library. Saved output is from the checked CPU run.\n"
      ],
      "id": "lift-splat-shoot-geometry-0"
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "source": [
        "import math\n",
        "from collections import defaultdict\n",
        "\n",
        "u, v = 70.0, 60.0\n",
        "cx, cy, focal = 50.0, 50.0, 100.0\n",
        "depths = ((8.0, 0.2), (10.0, 0.6), (12.0, 0.2))\n",
        "\n",
        "def lift(depth, pitch_degrees):\n",
        "    x = (u-cx)*depth/focal\n",
        "    y = (v-cy)*depth/focal\n",
        "    z = depth\n",
        "    angle = math.radians(pitch_degrees)\n",
        "    world_z = math.sin(angle)*y + math.cos(angle)*z\n",
        "    return x, world_z\n",
        "\n",
        "def splat(cell_width, pitch):\n",
        "    grid = defaultdict(float)\n",
        "    for depth, mass in depths:\n",
        "        x, z = lift(depth, pitch)\n",
        "        cell = (math.floor(x/cell_width), math.floor(z/cell_width))\n",
        "        grid[cell] += mass\n",
        "    assert abs(sum(grid.values())-1.0) < 1e-12\n",
        "    return [(cell, round(mass, 3)) for cell, mass in sorted(grid.items())]\n",
        "\n",
        "for width in (1.0, 2.0):\n",
        "    for pitch in (0.0, 5.0, -5.0):\n",
        "        print('width', width, 'pitch', pitch, 'cells', splat(width, pitch))\n",
        "base = lift(10.0, 0.0)\n",
        "tilted = lift(10.0, 5.0)\n",
        "print('depth10_forward_shift', round(tilted[1]-base[1], 4))\n",
        "assert base == (2.0, 10.0)\n"
      ],
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "width 1.0 pitch 0.0 cells [((1, 8), 0.2), ((2, 10), 0.6), ((2, 12), 0.2)]\n",
            "width 1.0 pitch 5.0 cells [((1, 8), 0.2), ((2, 10), 0.6), ((2, 12), 0.2)]\n",
            "width 1.0 pitch -5.0 cells [((1, 7), 0.2), ((2, 9), 0.6), ((2, 11), 0.2)]\n",
            "width 2.0 pitch 0.0 cells [((0, 4), 0.2), ((1, 5), 0.6), ((1, 6), 0.2)]\n",
            "width 2.0 pitch 5.0 cells [((0, 4), 0.2), ((1, 5), 0.6), ((1, 6), 0.2)]\n",
            "width 2.0 pitch -5.0 cells [((0, 3), 0.2), ((1, 4), 0.6), ((1, 5), 0.2)]\n",
            "depth10_forward_shift 0.0491\n"
          ]
        }
      ],
      "id": "lift-splat-shoot-geometry-1"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Sources\n",
        "\n",
        "- Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D: https://arxiv.org/abs/2008.05711\n"
      ],
      "id": "lift-splat-shoot-geometry-2"
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.12.14"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}