Olm Architecture

OpenLitterMap v5 — Architecture Reference

This is the canonical reference for OLM's v5 architecture. Read this before writing any backend code.

Core Principle: Every Tag Change Flows Through MetricsService

Nothing writes metrics directly. Every tag creation, edit, or deletion flows through a pipeline that ensures metrics, leaderboards, Redis, and the database stay in sync.

The Tag Pipeline

User/Admin/Teacher tags a photo
    ↓
AddTagsToPhotoAction::run($userId, $photoId, $tags)  ← wrapped in DB::transaction()
    ├── Resolves tags:
    │   ├── Object tags: uses category_litter_object_id (CLO ID), auto-resolves category
    │   ├── Custom-only: $tag['custom'] = true, $tag['key'] = "dirty-bench"
    │   ├── Brand-only: $tag['brand_only'] = true
    │   └── Material-only: $tag['material_only'] = true
    ├── Creates PhotoTag + PhotoTagExtraTags rows
    ├── Calls GeneratePhotoSummaryService
    │   ├── Builds summary JSON from PhotoTags (numeric ID keys)
    │   ├── Calculates XP via XpScore enum multipliers
    │   └── Writes photo.summary, photo.xp, photo.total_tags
    ↓
updateVerification() — WHO tagged it?
    ├── Trusted user → verified = ADMIN_APPROVED, fires TagsVerifiedByAdmin
    ├── Non-trusted user → verified = UNVERIFIED (0), fires TagsVerifiedByAdmin
    │   (Photo NOT on map, but user gets immediate leaderboard credit)
    ├── School student → verified = VERIFIED (1), does NOT fire event
    │   (Waits for teacher approval — safeguarding invariant)
    ↓
TagsVerifiedByAdmin event (fires for ALL non-school users)
    → ProcessPhotoMetrics listener
        → MetricsService::processPhoto($photo)
            ├── Computes fingerprint from summary JSON
            ├── If new (processed_at null): creates metric rows
            ├── If re-edit (processed_at set): computes delta
            ├── Upserts metrics table (5 timescales × 4 location scopes × 2 user modes)
            ├── Updates Redis (stats hashes, XP ZSETs, bitmaps, HLLs)
            └── Sets photo.processed_at, processed_fp, processed_tags, processed_xp

Key distinction: TagsVerifiedByAdmin fires for ALL non-school users at tag time. Trusted users also get verified = ADMIN_APPROVED (photo visible on map). Non-trusted users stay at verified = 0 (photo NOT on map, but user IS on leaderboard). Only school students' photos wait for teacher approval.

VerificationStatus Enum

enum VerificationStatus: int
{
    case UNVERIFIED = 0;      // Uploaded, no tags (also: tagged by non-trusted non-school user)
    case VERIFIED = 1;        // Tagged by school student (awaiting teacher)
    case ADMIN_APPROVED = 2;  // Approved by admin/trusted/teacher
    case BBOX_APPLIED = 3;    // Bounding boxes drawn
    case BBOX_VERIFIED = 4;   // Bounding boxes verified
    case AI_READY = 5;        // Ready for model training
}

Critical rule: Use ->value >= VerificationStatus::ADMIN_APPROVED->value (not == 2) when checking if a photo is public-ready. Photos at BBOX_APPLIED+ are also approved.

The enum is cast on the Photo model: 'verified' => VerificationStatus::class. Use ->value for >=/< comparisons, direct enum for === equality.

The Atomic Approve Pattern

Used everywhere: admin approve, teacher approve, batch approve.

$updated = Photo::where('id', $photo->id)
    ->where('is_public', true)
    ->where('verified', '<', VerificationStatus::ADMIN_APPROVED->value)
    ->update(['verified' => VerificationStatus::ADMIN_APPROVED->value]);

if ($updated > 0) {
    event(new TagsVerifiedByAdmin(
        photo_id: $photo->id,
        user_id: $photo->user_id,
        country_id: $photo->country_id,
        state_id: $photo->state_id,
        city_id: $photo->city_id,
        team_id: $photo->team_id
    ));
    rewardXpToAdmin();
}

Why atomic: Prevents double-processing. If two admins approve simultaneously, the WHERE clause ensures only one succeeds. The second gets $updated = 0 and no event fires.

Always pass all 6 named constructor args to TagsVerifiedByAdmin. Missing args cause listener failures.

MetricsService

Single writer rule. Only MetricsService writes to the metrics table and Redis metric keys. No other code may increment/decrement counters. Never use DB::table('metrics')->increment() or Redis::hincrby() directly.

processPhoto($photo)

• Computes fingerprint from summary JSON
• If processed_at is null: first processing → doCreate() (creates metrics + increments users.xp)
• If processed_at is set: re-processing → doUpdate() (computes delta, applies corrections to users.xp)
• Upserts metrics table rows: 5 timescales × 4 location scopes × 2 (aggregate user_id=0 + per-user user_id>0)
• Updates Redis via RedisMetricsCollector (inside DB::afterCommit)
• Sets photo.processed_at, processed_fp, processed_tags, processed_xp

recordUploadMetrics($photo, $uploadXp)

• Gated on $team->isSchool() check — school photos skip entirely (deferred to approval). The gate is a school team check, NOT an is_public check.
• Private-by-choice photos (non-school, is_public = false by user preference) still get immediate upload XP
• For non-school photos: writes 1 upload + upload XP to metrics/Redis, sets processed_at
• At tag time, processPhoto() routes to doUpdate() (delta-based) since processed_at is set

deletePhoto($photo)

• Must run BEFORE $photo->delete() — hard sequencing rule
• Reverses all metrics that processPhoto created
• Only runs if processed_at is not null
• Clears all processed_* columns
• Prunes zero-XP members from Redis leaderboard ZSETs

// CORRECT order
if ($photo->processed_at) {
    $this->metricsService->deletePhoto($photo);
}
$photo->delete(); // soft delete

// WRONG — metrics can't read a deleted photo
$photo->delete();
$this->metricsService->deletePhoto($photo); // too late

Photo Summary JSON

Every tagged photo has a summary JSON column — a flat array of tag entries, one per photo_tags row:

{
    "tags": [
        {
            "clo_id": 42,
            "category_id": 1,
            "object_id": 5,
            "type_id": 3,
            "quantity": 2,
            "picked_up": true,
            "materials": [1, 4],
            "brands": { "7": 1 },
            "custom_tags": [45]
        }
    ],
    "totals": {
        "litter": 3,
        "materials": 3,
        "brands": 1,
        "custom_tags": 0
    },
    "keys": {
        "categories": { "1": "alcohol" },
        "objects": { "5": "bottle" },
        "types": { "3": "beer" },
        "materials": { "1": "glass" },
        "brands": { "7": "heineken" }
    }
}

Structure: tags[] — each entry maps 1:1 to a photo_tags row. materials and custom_tags are ID arrays (set membership, qty = parent tag quantity); brands is an {id: quantity} map (independent quantities); totals.litter = sum of object quantities; keys is a reverse-lookup for display (empty sub-maps omitted). Generated by GeneratePhotoSummaryService from PhotoTag rows. If summary is null, the photo has no tags and cannot be approved.

PhotoTags (v5 Tag System)

photo_tags (
    id, photo_id,
    category_id,                  -- FK → categories (auto-resolved from object)
    litter_object_id,             -- FK → litter_objects
    category_litter_object_id,    -- FK → category_litter_object pivot (CLO ID)
    litter_object_type_id,        -- FK → litter_object_types (e.g., "beer" in "beer bottle")
    custom_tag_primary_id,        -- for custom-only tags (no category/object)
    quantity, picked_up,
    created_at, updated_at
)

photo_tag_extra_tags (
    id, photo_tag_id,
    tag_type,      -- 'material' | 'brand' | 'custom_tag'
    tag_type_id,   -- FK → materials / brandslist / custom_tags_new
    quantity, index,
    created_at, updated_at
)

v4 vs v5: v4 stored tags in 16+ separate category tables (smoking_id, food_id, etc on photos). v5 uses the unified PhotoTags table. Legacy v4 endpoints and the ConvertV4TagsAction shim have been removed (2026-03-01). Mobile now uses v3 endpoints with CLO format. Category FK columns on photos are deprecated.

Frontend sends CLO IDs: The web frontend sends category_litter_object_id (pre-resolved from the search index). Backend auto-resolves category_id from the CLO pivot. Category need NOT be sent separately.

No unique constraint on photo_tags: There is no unique constraint on (photo_id, category_litter_object_id, litter_object_type_id). Duplicate CLO+type combinations are possible (each row is a separate PhotoTag). Extra-tag deduplication (materials/brands) is handled via upsert within a single PhotoTag row, not across rows. Do NOT add a unique constraint expecting one row per (photo, CLO, type) combination.

getNewTags() conditional inclusion: UsersUploadsController::getNewTags() conditionally includes category and object only when both resolve (i.e., when both category_id and litter_object_id are non-null). Extra-tag-only PhotoTags (brand/material/custom-only) return category: null and object: null.

Upload error contract: UploadPhotoRequest::failedValidation() returns { success: false, error: <code>, message: <string>, errors: {...} }. resolveErrorCode() maps validation failures to typed string codes: no_exif, no_gps, no_datetime, duplicate, invalid_coordinates, validation_error. Mobile clients should read the error field for programmatic handling.

School Privacy — The is_public Gate

Hard rule: School team photos have is_public = false until a teacher approves them.

// PhotoObserver::creating() — automatic privacy
if ($photo->team_id) {
    $team = Team::find($photo->team_id);
    if ($team && $team->isSchool()) {
        $photo->is_public = false;
    }
}

// ALL public-facing queries MUST use this:
Photo::public()  // →where('is_public', true)
// or:
->where('is_public', true)

School pipeline: student uploads → is_public = false → NO upload metrics (deferred) → teacher reviews in Facilitator Queue → teacher approves → is_public = true + verified = ADMIN_APPROVED → TagsVerifiedByAdmin fires → processPhoto() → doCreate() handles full XP (upload + tag) in one pass + increments users.xp.

Admin queue never sees school photos (WHERE is_public = true excludes them). Teachers are the sole approvers for their team via the Facilitator Queue (3-panel admin-like UI).

User Photo Visibility

Users can control the public visibility of their photos independently of the school pipeline.

users.public_photos (boolean, default true): A per-user default. New photos inherit this value unless overridden.

Visibility precedence (highest to lowest):

1. School team — always is_public = false (enforced by PhotoObserver, cannot be overridden)
2. Explicit is_public param in upload request
3. User's public_photos default
4. Falls back to true

Private-by-choice photos (non-school, is_public = false by user preference):

• Hidden from map, clusters, and all public endpoints (same as any is_public = false photo)
• Upload metrics are processed immediately (unlike school photos)
• User appears on leaderboard with full XP credit

School photos (is_public = false enforced):

• Hidden from map AND upload metrics are deferred until teacher approval
• recordUploadMetrics() is skipped (school team check, not is_public check)

Per-photo toggle: PATCH /api/v3/photos/{id}/visibility — owner-only, requires auth:sanctum. Blocked for school team photos (403). PhotoObserver marks affected tiles dirty on is_public change so the map updates correctly.

Spatie Roles & Permissions

Laravel Permission 6, web guard.

Check with: $user->hasRole('admin') or $user->hasRole('superadmin')

Admin middleware checks: hasRole('admin') || hasRole('superadmin')

Nav.vue isAdmin includes 'superadmin' role (not just 'admin' and 'helper').

Redis Key Patterns

All keys use hash tags {...} for Redis Cluster compatibility:

{g}:stats               → HASH (global stats: uploads, tags, litter, xp, ...)
{g}:lb:xp               → ZSET (global XP leaderboard, user_id → xp)
{g}:hll                  → HyperLogLog (contributor count)
{c:ID}:stats             → HASH (country stats)
{c:ID}:lb:xp             → ZSET (country leaderboard)
{s:ID}:stats             → HASH (state stats)
{s:ID}:lb:xp             → ZSET (state leaderboard)
{ci:ID}:stats            → HASH (city stats)
{ci:ID}:lb:xp            → ZSET (city leaderboard)
{u:ID}:stats             → HASH (per-user stats: uploads, xp, litter)
{u:ID}:bitmap            → Bitmap (activity tracking)

Key builder: RedisKeys::global() → {g}, RedisKeys::country($id) → {c:$id}, etc.

rewardXpToAdmin() updates both MySQL users.xp AND Redis {g}:lb:xp sorted set.

Redis is a derived cache. Rebuildable from the metrics table. Never treat Redis as source of truth.

Soft Deletes

Photo model uses SoftDeletes trait. $photo->delete() sets deleted_at, does not remove the row. Eloquent auto-applies global scope excluding soft-deleted records. Raw queries need explicit whereNull('deleted_at').

Teams

Types: community and school. Stored in team_types table (resolve via $team->getTypeNameAttribute()). Do NOT hardcode type IDs — they vary between environments.

School teams enforce extra safeguarding: MasksStudentIdentity trait masks student names as "Student N" (deterministic, based on team_user.id join order). Student identity never exposed publicly.

School teams must NOT be is_trusted — trust bypasses teacher approval entirely.

Key methods: $team->isSchool(), $team->isLeader($userId), $team->hasSafeguarding()