Site Storage Heatmap

Site Storage Heatmap

Generate an interactive HTML site map with storage sizes and hot/cold activity heatmaps for a SharePoint site, then navigate the user to the output.

Steps

1. Get the Current User's First Name

• Call get_user_info(query="@currentUser") to resolve the current user.
• Extract the first name from the resolved display name (e.g., "Adam Harmetz" → "Adam").
• The output filename will be: {FirstName}-Site-Storage-Heatmap.html

2. Discover All Containers

• Call discover_sharepoint_lists with filterType="all" and includeHidden=false to get every list, library, and page library on the site.
• Categorize each container:
  • Document Libraries: templateType 101
  • Lists: templateType 100
  • Site Pages: templateType 119
  • Calendar: templateType 106
• Record each container's id, title, itemCount, and serverRelativeUrl.

3. Collect File-Level Data from Each Non-Empty Library

• For every document library (templateType 101) with itemCount > 0, call list_items with:
  • viewFields: ["FileLeafRef", "File_x0020_Size", "File_x0020_Type", "Modified"]
  • itemType: "files", recursive: true, rowLimit set to the library's itemCount
• For Site Pages (templateType 119), also query with viewFields: ["FileLeafRef", "Modified"] to get page modification dates.
• Parallelize all independent list_items calls.

Large sites (≥ 100 libraries or any library with ≥ 5,000 items)

Naïvely parallelising every list_items call against a large site will hit throttling (HTTP 429) and balloon memory because every file row sits in the model state.

• Batch the parallel fan-out at no more than ~10 concurrent list_items calls. After each batch, aggregate the results into the running totals via execute_code with outputDataRef: true so the raw rows are released from model state. Never hold all libraries' file rows in memory at once.
• Per-library paging. If a library reports itemCount > 5,000, fetch it in pages of 5,000 (the SharePoint list view threshold) rather than one giant rowLimit. Aggregate per-page into running totals — do not retain the per-page raw arrays after they are summed.
• On 429 / throttling: back off, retry the affected library once with a smaller rowLimit, and continue. If retry fails, record the library as Partial data — throttled and surface it explicitly in the HTML output (a yellow badge on that library card) so the consumer knows the totals are an undercount, not zero.
• Progress reporting. Every 10 libraries processed, post a short progress line (Processed 30 of 142 libraries…) so a multi-minute run doesn't look frozen.
• Never silently drop a library. Every container from Step 2 must appear in the final HTML output, even if its row is annotated (no data) or (throttled).

4. Aggregate Per-Library Stats

For each library, compute:

• Total size (sum of File_x0020_Size in bytes)
• File count
• File type breakdown: group by File_x0020_Type → { ext: { count, size } }
• Recency buckets based on Modified date relative to today:
  • Hot: modified ≤ HOT_DAYS ago (default 7)
  • Warm: modified ≤ WARM_DAYS ago (default 30)
  • Cool: modified ≤ COOL_DAYS ago (default 90)
  • Cold: modified > COOL_DAYS ago
• Track both file count AND storage per recency bucket.

Configurable bucket thresholds

The 7 / 30 / 90 day defaults are conventional but not always right — a fast-moving project site may want 1 / 7 / 30, a records archive may want 30 / 180 / 365.

• Honour explicit overrides in the user's prompt. Examples that must work:
  • "heatmap with hot=1 day, warm=7, cool=30"
  • "use 30/90/365 buckets"
  • "only consider files older than a year as cold"
• Validate: HOT_DAYS < WARM_DAYS < COOL_DAYS, all positive integers. If the user gives invalid values, ask once for correction; do not silently re-order.
• Always render the thresholds used in the HTML output — a small legend line under the heatmap such as Hot ≤ 7d · Warm ≤ 30d · Cool ≤ 90d · Cold > 90d — so the consumer can tell at a glance what "hot" means in this report.

For lists (no file sizes), classify each list as hot/warm/cool/cold based on the list's lastModified date from the discovery response. Count list items per recency bucket.

5. Compute Site-Wide Rollup

Sum across ALL containers (libraries + lists + pages):

• Grand total items (files + list items + pages)
• Total storage (libraries only)
• Aggregate recency: total hot / warm / cool / cold counts
• Separate recency breakdowns for: Document Libraries, Lists, Site Pages

6. Generate the Interactive HTML

Build a self-contained HTML file with embedded CSS and JavaScript containing:

Root Banner

• Site name, total item count, container count, page count
• "Total Library Storage: X" badge