Kb AI Review Lite

KB AI Review Lite

Goal

Perform a lightweight AI validation pass on one or more KB files and store the result as metadata only.

This sample intentionally focuses on one pattern: create required list columns when absent, then populate them.

---

Use Cases

Use this skill when the user asks to:

• run a KB review
• generate review notes
• add missing review columns
• mark completion and next review timing

Prompt examples:

• "review these KB files"
• "run AI review"
• "create AI review columns and summarize"

---

Guardrails

• Do NOT modify the original KB document content.
• Do NOT create or upload draft Word files.
• Do NOT publish content.
• Only update the fields defined in this skill.
• Return results in Markdown.

---

Required Columns (Create If Missing)

Before processing, inspect library schema and ensure all columns below are present.

1. AI Review State

• Type: Choice
• Internal name: AIReviewState
• Choices: Review Complete, Not Run, Published
• Recommended default: Not Run

2. AI Review Summary

• Type: Note (multi-line text)
• Internal name: AIReviewSummary
• Required capacity: long text
• When creating in a document library, you MUST set UnlimitedLengthInDocumentLibrary: true in the field schema at creation time. The default is false and caps input at 255 characters, which will cause writeback failures. This is not optional.

3. AI Draft Payload

• Type: Note (multi-line text)
• Internal name: AIDraftPayload
• Content format: Markdown
• Required capacity: long text
• When creating in a document library, you MUST set UnlimitedLengthInDocumentLibrary: true in the field schema at creation time. The default is false and caps input at 255 characters, which will cause writeback failures. This is not optional.

4. Completed On

• Type: DateTime
• Internal name: CompletedOn

5. Reviewed By

• Type: Text
• Internal name: ReviewedBy
• Value guidance: current user or active agent display name

6. Review Score

• Type: Number
• Internal name: ReviewScore
• Range: 0 to 100

7. Failure Reason

• Type: Note (multi-line text)
• Internal name: FailureReason
• Keep blank on success; set concise reason on failures

8. Next Review Date

• Type: DateTime
• Internal name: NextReviewDate

9. KB Category

• Type: Choice
• Internal name: KBCategory
• Choices: Security, Infrastructure, General, Applications, Other
• Recommended default: Other

Create missing columns with create_or_update_list and avoid duplicates.

When creating these columns for a document library, set them to unlimited-length multi-line mode.

---

Recertification Cadence

Set NextReviewDate using category-based intervals:

• Security -> CompletedOn + 30 days
• Infrastructure -> CompletedOn + 60 days
• General -> CompletedOn + 90 days
• Applications -> CompletedOn + 90 days
• Other -> CompletedOn + 90 days

If the user supplies an override cadence, apply it instead.

---

Accuracy Baseline

Assess KB content using category-specific sources:

• Security: Validate against Microsoft MSRC, vendor CVE databases, and internal security policies.
• Infrastructure: Validate against vendor documentation (AWS, Azure, Cisco, etc.), public runbooks, and internal standards.
• Applications: Validate against application vendor docs, public APIs, and internal implementation standards.
• General: Validate for internal consistency and alignment with internal policy.
• Other: Validate for internal consistency only.

When using web sources, prioritize official vendor documentation first. Include source URLs for any critical correction.

---

Output Quality Gates

The review is considered weak and must be regenerated if any of the following occur:

• Findings are generic (for example: "validate details" or "recheck meanings") without concrete wrong and correct values.
• Update Required: Yes is set but no specific section identifiers are listed.
• Fewer than 5 concrete findings are reported when a document has obvious factual issues.
• Critical operational or safety issues are found but not labeled as High severity.
• No evidence source is provided for material factual corrections.
• Update Required: No is returned with Findings: 0 for a parameter-heavy document (URLs, versions, timeouts, ports, phone numbers, thresholds, procedures) without explicit claim-by-claim verification evidence.
• Summary says Update Required: No but payload still contains open questions that imply unresolved factual uncertainty.

Minimum detail requirements when updates are needed:

• Summary must include at least 5 concrete findings; target 8+ for heavily incorrect documents.
• Each finding must include: section identifier, incorrect claim, corrected value, and severity (High, Medium, Low).

Minimum detail requirements when no updates are claimed:

• If Update Required: No, include Verified Claims: <count> in summary.
• Verified Claims must be at least 8 for operational KBs.
• Payload must include a ## Verification Evidence section with at least 8 verified claim rows and sources.
• If these conditions are not met, treat the review as failed quality gate.

---

Payload Contract (Strict)

When Update Required: Yes, AIDraftPayload MUST include all sections below in this exact order:

1. ## Findings
2. Markdown table with columns exactly: ID | Section | Incorrect Claim | Correct Value | Severity | Evidence
3. ## Proposed Updates
4. ## Suggested Replacement Text
5. ## Open Questions
6. ## Risks

Additional strict rules:

• Evidence must contain a source URL or internal reference per finding.
• For High severity findings, evidence is mandatory (never blank).
• If the document has obvious factual conflicts, include at least 5 finding rows.
• Do not output ## Proposed Updates before ## Findings.
• Do not mark review complete if this contract is not met.
• If Update Required: No, include ## Verification Evidence after ## Findings and before ## Proposed Updates.

## Verification Evidence table format when Update Required: No:

Claim | Verified Value | Source | Result

Result must be Verified for each row.

---

Process

Step 1 - Resolve Target and Schema

• Determine target library and files (selected items, named files, or full library).
• Run get_list_schema to inspect column definitions.
• If an item is already Published, skip it unless re-review is explicitly requested.

Step 2 - Provision Missing Columns

• For each required column, check by internal name first.
• Create only missing columns using standard column creation. Do not create duplicates.
• Always create columns as standard columns, not autofill or content extraction columns. Column values are written explicitly by this skill. Never prompt the user to choose between column modes.

Before creating any columns, set an internal flag: columns_created = false

If any column was missing and was created during this run:

• Set columns_created = true

After all provisioning is complete, evaluate the flag:

If columns_created = true:

• Do NOT proceed to Step 2.1 or Step 3.
• Do NOT touch any file metadata.
• Do NOT set AIReviewState or FailureReason on any item.
• Display the following setup notice and stop all processing:

---

✅ All review columns have been created with the correct names and types.

⚠️ One manual step is required before the review can run.

Two columns need their text length limit changed in SharePoint:

1. Go to Library Settings → Columns
2. Click AI Review Summary
   • Find Allow unlimited length in document libraries
   • Select Yes → Save
3. Click AI Draft Payload
   • Find Allow unlimited length in document libraries
   • Select Yes → Save

Once both settings are saved, run the review again. All other columns are ready.

---

This is a one-time setup step per library. It will not be required on subsequent runs.

If the user replies in the same session confirming the column settings have been changed, proceed directly to Step 2.1 without re-provisioning or re-evaluating the flag.

If columns_created = false:

• All columns already existed before this run.
• Proceed directly to Step 2.1.