# Source ingestion — the retrieval ladder Getting *all* the sources is most of the work. This is the order to try things, and what fails. --- ## 1. Email threads Use the direct WorkIQ tools before the agent — they're faster and deterministic. ``` workiq_search_emails query + startDate/endDate workiq_get_email emailId, includeAttachments: true ``` The originating ask is often buried months back in a thread with a different subject line. Search on the customer domain, not just the subject. **Read the recipient list.** It tells you who the stakeholders are, who is cc'd (observers, often the real decision makers), and which side of the house each person sits on. --- ## 2. Teams meeting transcripts ``` workiq_list_meetings startDate / endDate workiq_list_meeting_transcripts joinUrl | onlineMeetingId | calendarEventId workiq_get_meeting_transcript onlineMeetingId + transcriptId ``` ### The failure you will hit **If the customer organised the meeting, the transcript lives in their tenant and you get HTTP 403.** ``` {"error":"meetings_auth_required","status":403, "message":"Meeting tools require delegated scopes such as OnlineMeetings.Read and OnlineMeetingTranscript.Read.All"} ``` This is not a permissions bug you can fix. The recording is in the customer's Microsoft 365, not yours. Options: 1. Ask the customer to send it (they often will — and often already have) 2. If someone **on your side** recorded, their OneDrive `Recordings` folder has it — retrieve via a recap link (below) 3. Check whether it arrived as an **email attachment** (below) ### Retrieving from a Teams recap / OneDrive link A `teams.microsoft.com/l/meetingrecap?...` URL contains a `fileUrl` parameter pointing at the real file. Extract it, then: ```powershell workiq.cmd ask --file-urls "" -q "..." ``` `workiq_resolve_m365_link` converts sharing URLs into canonical Graph IDs when you need them. --- ## 3. Email attachments (transcripts often arrive this way) This is the awkward one. `workiq_get_email` with `includeAttachments: true` tells you the attachment **exists** and its size, but does not give you the bytes. Ladder, in order: **a. WorkIQ ask** — often returns only indexed metadata plus the first line for large attachments. Worth one attempt; don't fight it. **b. Outlook COM** — frequently unavailable: ``` COM failed: Cannot complete the operation. You are not connected. ``` Happens with the new Outlook client or no MAPI profile. Don't invest here. **c. OWA → "Save to OneDrive"** ← the one that works Browser automation against Outlook on the web. Open the message, right-click the attachment, choose **Save to OneDrive**. The attachment menu offers: `Preview · Edit in browser · Edit in Word desktop app · Save to OneDrive · Copy · Download`. `Download` fails under automation (`Pass { acceptDownloads: true } when creating your browser context`). **Save to OneDrive avoids downloads entirely** and puts the file somewhere supported tools can reach. **d. Local OneDrive sync** ← fastest, check this first next time Once it is in OneDrive it syncs to disk, usually under `OneDrive - \Attachments\`. Search local disk before anything clever: ```powershell Get-ChildItem "$env:USERPROFILE\OneDrive - Contoso" -Recurse -Filter '*keyword*.docx' ``` Files On-Demand placeholders show `Attributes: Archive, ReparsePoint` and a small on-disk size — they still read fine, the read triggers a hydrate. **Note:** the search index lags several minutes behind a newly saved file. `workiq_search_files` may return nothing while `workiq_get_recent_files` already shows it. Use recent-files for anything just created. --- ## 4. Extracting text from a .docx transcript A `.docx` is a ZIP. `word/document.xml` holds the text. Use `scripts/extract-docx-text.py`. Do **not** try to reason over the raw XML, and do not use local extraction for documents that live only in SharePoint/OneDrive — use WorkIQ Ask for those. Local extraction is for files genuinely on disk. Transcript text is small even when the file is large (a 3 MB docx was 56 KB of text — the rest was embedded images). --- ## 5. Live captions during the meeting If you capture Teams Live Captions yourself, know the failure mode: **Each written line is a cumulative snapshot of the entire visible captions window, not the new text.** The file grows quadratically — a 75-minute meeting produced 13 MB / ~4,800 lines. Never read the whole file. ```powershell Get-Content $file -Tail 1 # current window only ``` Then diff against a stored marker to find genuinely new content. **Captions are lossy.** They mis-attribute speakers (an unattributed device shows as "Speaker 1" / "Orador 1"), drop or garble numbers, and truncate. Treat everything from captions as 🟡 confidence until an official transcript confirms it. **Purge raw captions when the meeting ends.** Keep the extracted insights, delete the transcript body. --- ## 6. Screenshots Screenshots pasted during a reverse demo are high-value and easy to lose. They capture: - the actual navigation structure and terminology (use their words) - real KPI values and formulas you can reproduce - what's an iframe vs native - load times and empty states Log what each one shows **in the session**, not later. You will not remember which of 20 screenshots had the bonus formula. --- ## 7. Speaker attribution Resolve unattributed speakers by cross-reference, not guesswork: > `Orador 1: [explains the BI platform]` > `Lead SE: "Is that standard, ? Is that standard for your BI?"` > `Orador 1: "Yes, correct."` Someone addressing the speaker by name in the next turn is your proof. Record the resolution and the evidence. --- ## Ingestion completeness checklist - [ ] Full email thread, both directions, including attachments - [ ] Official transcript (or documented why it's unavailable) - [ ] Your own captions, insights extracted, raw purged - [ ] Every screenshot described - [ ] Internal debrief - [ ] Prior sessions on the same account - [ ] Speaker attribution resolved with evidence - [ ] Conflicts between sources listed explicitly