UTeM · Tuah
RAG Studio
Plugin → web app

Migration Checklist

Every capability from the ChatGPT plugin export, mapped to its status here.

Implemented

  • done
    JSON import & input detection
    Distinguishes RAG array, wrapped records and raw crawl envelope.
  • done
    Exact eleven-key schema validation
    Keys, types, snake_case IDs, lang, 3–5 variants, 4-field metadata, Malaysia-time crawled_at_my_tz, 30–70-word search_text, source links.
  • done
    Audit ledger
    Every input entry is approved / hold / excluded with reasons; merge mappings retained.
  • done
    06 270 phone exclusion
    Same regex as the original crawler; +606 229 2182 and emails preserved.
  • done
    Website crawler (port of crawl_website.py)
    Depth 4 / 300 attempts / 5 redirects / 0.5 s / 20 s / 8 MiB, robots, www scope, EN+MS, document-route filter, priority, checkpoints.
  • done
    Deduplication
    Duplicate IDs, identical answers, near-duplicate questions with reviewer-controlled merge.
  • done
    RAG JSON export
    Approved-only top-level array, web_<entity>_rag_<DDMMYY>.json, KL date.
  • done
    Q&A workbook export
    Real XLSX: one Q&A sheet, 4 text columns, frozen header, filter, reopened and reconciled 1:1.
  • done
    Word audit summary
    Genuine .docx, ≤200 words, built from audit evidence only.
  • done
    Bilingual RAG generation (Stage 2)
    Lovable AI converts each crawled page under the original SKILL.md, DBP and Fact & Figure rules. Metadata/source_url/lang come from the fetch; every record must quote verbatim evidence or it is held; 06 270 numbers scrubbed.

Partially implemented

  • partial
    Vector readiness
    Deterministic checks + local BM25 retrieval with must_match. No embeddings or deployed vector-store test.
  • partial
    Knowledge Manager persistence
    Stored in this browser (IndexedDB). Shared, multi-user storage needs Lovable Cloud.
  • partial
    Crawler runtime
    Runs from the open browser tab; closing it stops the crawl (checkpoint kept). No JavaScript page rendering.
  • partial
    Workbook citation rule
    Original instructions conflict; resolved by an explicit user toggle in Export Center.
  • partial
    Audit Word review
    Content verified; rendered visual layout review is not automated.

Missing dependencies

  • partial
    Semantic evidence review
    Verbatim-evidence check is automatic; contradiction/currentness review across pages still needs a human reviewer.
  • missing
    DBP Malay sign-off
    DBP rules are applied during generation, but authoritative DBP terminology lookup has no service adapter; human sign-off required.
  • missing
    Generation from uploaded legacy RAG/XLSX
    Re-converting existing records (e.g. legacy FAIX entries lacking provenance) is not supported; they stay held.
  • missing
    XLSX input import
    Uploading XLSX as source input is not yet supported (JSON only).
  • missing
    Staff directory / glossary snapshots
    Not in the export; cannot be recreated.
  • missing
    Fact & Figure label verification by rendering
    Needs a headless browser/screenshot service.

Required integrations

  • done
    Lovable AI
    Enabled; used for bilingual generation.
  • missing
    Lovable Cloud database
    Cloud is enabled, but datasets still live in the browser; shared storage and sign-in are not built yet.
  • missing
    Vector store + embeddings
    Your Development Vector Store (provider and credentials not specified).
  • missing
    Rendering service (optional)
    For JavaScript pages and statistic cards.

Validation & testing status

  • done
    Automated tests (14 passing)
    Phone rule, document routes, URL normalisation, cfemail, robots, parser contacts, crawl engine with redirects/robots/scope, schema rules, FAIX sample (53 entries held), dedup, naming, XLSX round-trip.
  • done
    Live AI generation
    One real generation on FAIX page text: 3 records, 0 schema issues, 06 270 fax removed, +606 229 2182 kept.
  • partial
    Live crawl
    Engine tested with simulated site responses only; a live crawl of a UTeM site has not been run in this session.
  • missing
    Retrieval quality
    No deployed vector-store or Playground test.