# Troubleshooting — every failure hit during the reference build

Ordered roughly by how much time each one costs if you do not know about it.

---

## `pac` CLI gives confidently wrong answers

| Command | Symptom | Trust instead |
|---|---|---|
| `pac application list --installState` | 33 apps appear in **both** Installed and NotInstalled | Org UI app list |
| `pac solution list` | Returns 3 solutions for a fully-deployed org | Org UI app list |
| `pac copilot list` | All agents "Published / Provisioned" when 37 are Draft | Copilot Studio Agents list |
| `pac admin list` | Type = Sandbox after successful Production conversion | PPAC environment hub |
| `pac admin list` | Omits environments entirely | Only shows envs **with a Dataverse database** |
| `pac application install` | "already installed. Proceeding with update" then success, imports nothing | Org UI app list |

**Rule: `pac` is fine for *doing* things, unreliable for *checking* things.**

---

## Installs that appear to hang

`pac application install` frequently never returns even after the app fully deploys.
Project Operations was verified live in the org UI while the CLI kept polling for another
50 minutes.

At exactly 60 minutes it hard-errors:

```
Error: Failed to install application within maximum timeout of 60 minutes
```

That error does **not** mean the install failed. Check the org UI.

**Do not declare an install dead before 60 minutes.**

---

## Agent admin pages stuck on "Loading"

**Cause:** connection references are unbound. See `02-connections-and-agents.md`.

There is no error message. The page just spins. This is the single highest-value thing in
this skill.

---

## `PvaPublish` returns 409 Conflict

```json
{"error":{"code":"0x80040265","message":"{\"statuscode\":409,\"statusdescription\":\"Conflict\",
\"errors\":[{\"message\":\"Exception of type 'Microsoft.Dynamics.PowerVA.Plugins.Exceptions.ExternalServiceException' was thrown.\"}]}"}}
```

Accompanied by: Copilot Studio agent status **"Setting up…"** and a **disabled** Publish button.

Server-side provisioning state. Not fixable client-side. Did not clear after 25+ minutes.
See the publish section in `02-connections-and-agents.md`.

---

## Theme applied but nothing changes

Three possible causes, in order of likelihood:

1. You used the **legacy `theme` entity**. It does nothing in modern UCI. See
   `03-branding-and-polish.md`.
2. You set `CustomThemeDefinition` but not **`EnableDesignLanguageThemeSystem = true`**.
   That is the master switch and it defaults to false.
3. You are checking the **app-selector page** (`pagetype=apps`), which has its own fixed
   chrome. Check inside a real app.

---

## `webresourceset` not found

```
The entity "webresourceset" cannot be found.
```

The entity logical name is **`webresource`**. `webresourceset` is the OData collection name and
is not valid in `Xrm.WebApi.createRecord`.

---

## "undeclared property 'SettingDefinitionId'"

Lookup binds are **lowercase**:

```js
'settingdefinitionid@odata.bind': '/settingdefinitions(<guid>)'   // correct
'SettingDefinitionId@odata.bind': '...'                            // fails
```

To discover the correct navigation property for any entity:

```js
const r = await fetch(base +
  "/api/data/v9.2/EntityDefinitions(LogicalName='organizationsetting')?$select=LogicalName" +
  "&$expand=ManyToOneRelationships($select=ReferencingEntityNavigationPropertyName,ReferencedEntity)",
  {headers:{Accept:'application/json'}});
```

---

## "You cannot create system or default themes"

You passed `type: false` when creating a `theme` record. Omit `type` entirely.

(Moot anyway — the legacy theme entity has no visual effect. See above.)

---

## Unhandled server exception on `main.aspx`

Adding arbitrary query parameters (e.g. a cache-buster `&t=123456`) to a `main.aspx` URL
produces an error page:

```
Error Details: An unhandled exception occurred during the execution of the current web request.
```

Use the plain URL.

---

## Application-user dropdown shows only "None"

The Case resolution settings page does not refresh its application-user list when you navigate
directly to the settings control URL.

**Fix:** go to the parent **Case Management Agent** page and click
**Manage settings for case resolution**. The dropdown populates correctly.

---

## PPAC shows the wrong tenant

If you are a Microsoft employee, PPAC silently opens in your **corporate** tenant. You will see
corporate environments and conclude the Caldova environment does not exist.

Check the account avatar. Get the Caldova tenant ID from the public openid-configuration
endpoint and force PPAC to it.

Related: **`az` CLI is very likely signed into your corporate tenant too.** Check with
`az account show`. Use `pac` for anything that must land in the demo tenant.

---

## Cross-origin iframe failures in admin UIs

```
Failed to load resource: net::ERR_CONNECTION_CLOSED
  @ https://<envhash>.environment.api.powerplatform.com/...
```

The Power Platform admin widgets embedded in model-driven apps are hosted on island-cluster
domains and intermittently fail to load **in the browser** even when the host is reachable
from PowerShell.

**Workaround:** do the operation via `Xrm.WebApi` instead of the widget wherever possible.

---

## Coachmark / teaching-bubble dialogs block automation

PPAC shows onboarding coachmarks ("Explore Power Platform Inventory") that intercept pointer
events and cannot always be dismissed by clicking Next/Dismiss.

**Workaround:** reload the page, then remove them from the DOM:

```js
document.querySelectorAll('[role="dialog"]').forEach(d => {
  if (/Explore Power Platform Inventory|1 of 2/.test(d.innerText)) d.remove();
});
```

---

## Do not clear cookies to bust a cache

It signs you out of the tenant and you will need to re-authenticate interactively. A normal
reload is almost always enough. If content is genuinely cached server-side, waiting is the fix.

---

## MIP-protected PDFs cannot be parsed locally

The Caldova setup guides are MIP-protected. `markitdown`, `PyMuPDF` etc. return only a
protection stub.

**Use WorkIQ Ask with the SharePoint URL:**

```powershell
& "$env:USERPROFILE\.scout\bin\workiq.cmd" ask `
  --file-urls "https://<tenant>-my.sharepoint.com/personal/.../STEP 1 - ....pdf" `
  -q "Summarize the setup steps, preserving all URLs, settings and values."
```

It refuses verbatim reproduction but will produce a faithful paraphrase with values intact.

---

## Solo-only app provisioning error

```
The CDS app 'X' cannot be provisioned with other apps.
```

Contact Center, M365 Copilot – Service Solution, Guides and Remote Assist each require their
own dedicated environment. Remove them from the multi-select.

---

## `competitor` entity has no `description` field

```
Could not find a property named 'description' on type 'Microsoft.Dynamics.CRM.competitor'.
```

`competitor` supports `name` but **not** `description`. A `$select` including it returns
HTTP 400 and, if your code assumes success, silently yields `undefined` counts.

Always surface OData query failures explicitly rather than letting them fall through as
empty arrays — `verify-environment.js` does this via an `ERROR` key.

---

## Customer Insights install path is not discoverable

Three separate obstacles, all hit in sequence:

**1. `pac` refuses**
```
Error: This application utilizes a Custom Install Experience through a Single Page
Application. Install for such apps is not supported by this API.
```

**2. PPAC per-environment Applications page crashes**
`/manage/environments/{envid}/applications` ->
`Error: Cannot read properties of undefined (reading 'toString')`

**3. The working surface is not linked from the environment**
PPAC -> Products -> Dynamics 365 apps -> "Dynamics 365 Marketing Application" card ->
Manage -> Open -> redirects to `https://public-usa.mkt.dynamics.com/fre/cio/manage`

Bookmark that final URL.

---

## CI Journeys and CI Data install SERIALLY

Starting the CI Journeys install immediately disables the CI Data Install link
(`cursor: not-allowed`). You cannot run both at once. Wait for the first to complete.

---

## CI portal Install controls are `<a>` not `<button>`

```js
page.locator('button').filter({ hasText: /^Install$/ })   // times out
page.locator('a').filter({ hasText: /^Install$/ })        // works
```

Disabled state is signalled only by inline `style="... cursor: not-allowed"` - there is no
`disabled` attribute and no `aria-disabled`.

Similarly, PPAC app tiles are Fluent `[role="group"].fui-Card`, not grid rows. Row-based
locators return nothing.

---

## DynamicsMKT_SendOptimization does not mean Marketing is installed

A fresh environment commonly contains the solution
`DynamicsMKT_SendOptimization` ("Dynamics Marketing Send Optimization"). It is a small
supporting solution, not the app. Check the CI provisioning portal for the real state.

---

## CI Data does NOT need your own Azure Data Lake

Earlier assumption, now corrected by testing: the CI - Data install dialog asks
"Save output data to" and defaults to **Customer Insights managed storage**, which is
Microsoft-managed. No Azure subscription needed for output.

Your own Azure Data Lake Gen2 is the alternative option, for customers who require the
data in their own tenant.

Ingestion sources are configured separately after install; Power Query and Dataverse
both work without Azure.

---

## CI Journeys install takes 3+ hours and the portal lags

Measured Aug 2026 on a Production environment: **3 hours 11 minutes** end to end. At the 2h10m mark the
provisioning portal still showed `Installing` even after the
`Customer Insights - Journeys` app module already existed in the org.

Do not watch the portal. Poll Dataverse:
  - marketing solution count climbing, with a recent `installedon`  -> healthy
  - `Customer Insights - Journeys` present in `appmodules`          -> effectively landed
  - total visible solutions went 110 -> 228 across this install

Same lesson as the app installs: the status surface is less trustworthy than the org.


---

## 🔁 Endless Windows sign-in prompts for the demo tenant

**Symptom.** A **native Windows** credential dialog (not a browser page) keeps appearing asking you
to sign in to the Caldova tenant. It returns every time the agent host starts, and signing in never
makes it stop.

**Cause.** A **Dataverse MCP server registered against the demo org** — e.g.

```json
"dataverse-caldova": {
  "command": "node",
  "args": ["…/@microsoft/dataverse/bin/dataverse.js", "mcp",
           "https://<your-org>.crm.dynamics.com/"]
}
```

Your workstation is Azure AD-joined to **your corporate tenant**, but the CDX demo tenant is a
**completely different tenant**. On launch the Dataverse CLI asks **WAM**
(`Microsoft.AAD.BrokerPlugin`) for a token; WAM cannot obtain one silently for a foreign tenant, so
it raises the OS sign-in dialog. The token never persists, so **it re-prompts on every start**.

### Confirm it in four commands

```powershell
# 1. Is a Dataverse MCP pointed at the demo org?
Get-CimInstance Win32_Process -Filter "Name='node.exe'" |
  Where-Object { $_.CommandLine -match 'dataverse\.js mcp' } |
  ForEach-Object { if($_.CommandLine -match 'mcp (https://[^/ ]+)'){ "$($_.ProcessId)  $($Matches[1])" } }

# 2. Which tenant does THIS DEVICE belong to?
dsregcmd /status | Select-String 'AzureAdJoined|TenantName|TenantId|Executing Account Name'

# 3. Is there a cached token for the demo tenant? (a missing cache = auth never succeeded)
Get-ChildItem "$env:LOCALAPPDATA\.IdentityService" -Filter 'msal_*.cache'

# 4. Is WAM running? (it is what raises the OS-level dialog)
Get-Process Microsoft.AAD.BrokerPlugin -ErrorAction SilentlyContinue
```

Two decisive tells:

- **The MSAL cache folder holds a `msal_<corp-tenant-guid>.cache` and nothing for the demo tenant.**
  That proves the demo-tenant auth has never completed, not merely expired.
- **The server's tools are absent from the agent's tool list** even though the server is registered
  *and* enabled. It spawns, blocks on auth, and never finishes initializing — so it has been dead
  weight the whole time while still generating prompts.

Cross-check the process start time against when the prompt appeared; in the verified case the
process started at 08:59:58 and the prompt was reported at 09:01.

### Fix

**Either remove the registration, or give the CLI an auth profile — but do not leave it half-wired.**

> **The root cause is a missing auth profile, not cross-tenant impossibility.** Cross-tenant
> Dataverse MCP works fine *when the CLI has a persisted profile for that org*. On the verified
> machine, three sibling servers were registered and one of them (`potbelly`) pointed at an org in a
> **completely different tenant** from the corporate one — and it worked, because
> `dataverse auth list` held a matching profile. The demo org had **no profile at all**, so the CLI
> fell back to the corporate WAM identity and hit `AADSTS90072` forever.
>
> Diagnose it in one comparison — every registered MCP org should have a matching row here:
>
> ```powershell
> node "$env:APPDATA\npm\node_modules\@microsoft\dataverse\bin\dataverse.js" auth list
> ```

Pick one:

**A — you do not need MCP for this tenant (recommended for a demo build).** Remove the registration
(below). Every Dataverse change in this playbook is made through the browser against the Web API
(`/api/data/v9.2/…`) with the session you are already signed into, so you lose nothing.

**B — you want the MCP tools.** See *"Wiring a Dataverse MCP server to a demo tenant, in the right
order"* further down. **Create the auth profile first, register second.** Doing it the other way
round is exactly what produces the endless prompts.

> ### ⚠️ `enabled: false` is NOT enough — it does not stop the prompts
>
> Verified the hard way. Setting `permissions.servers["dataverse-<demo>"].enabled = false` in Scout
> settings **gates tool exposure, not process spawning.** The host still launches the server on
> every new session, it still attempts the cross-tenant token, and **you still get the prompt.**
>
> The tell is in `~/.scout/m-diagnostics.jsonl`, timestamped to the moment the dialog appears:
>
> ```
> {"level":"warn","category":"SessionManager",
>  "message":"MCP server \"dataverse-<demo>\" failed to connect", …}
> ```
>
> After disabling, prompts continued on a ~30–90 minute cadence (once per new session), while
> `Get-CimInstance Win32_Process` showed **no** surviving process for that org — because the spawn
> fails auth and exits before you can catch it.

The fix that actually works — **delete the entry from `~/.scout/m-mcp-servers.json`**:

```powershell
$f="$env:USERPROFILE\.scout\m-mcp-servers.json"
Copy-Item $f "$f.bak" -Force                       # always back up first
$j = Get-Content $f -Raw | ConvertFrom-Json
$j.servers.PSObject.Properties.Remove('dataverse-<demo>')
$j | ConvertTo-Json -Depth 12 | Set-Content $f -Encoding UTF8
# verify only that one key went, and that the rest kept their command/args/tools
(Get-Content $f -Raw | ConvertFrom-Json).servers.PSObject.Properties.Name
```

Then:

- **Stop the live process by PID** — no config change kills one that is already running.
- **Restart the host.** Sessions already open have spawned it; removal only takes effect for new ones.
- Optionally enable the **`authCooldown`** experiment, which damps repeat interactive prompts.

> **You lose nothing.** Every Dataverse change in this playbook is made through the browser against
> the Web API (`/api/data/v9.2/…`) with the session you are already signed into. The MCP server was
> never the working path — it only looked like one because it was registered.

### Related hygiene

Count the Dataverse MCP instances while you are in there. Duplicates accumulate across restarts —
the verified case had **3 copies each** of two servers (~1 GB of node processes), each one an extra
chance to prompt.

### The single most useful diagnostic: the host's own MCP log

Do not theorise about which app is prompting. Two sources name it outright.

**1. The AAD operational log names the tenant and the client application:**

```powershell
Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-AAD/Operational'
                                StartTime=(Get-Date).AddHours(-6)} -MaxEvents 800 |
  Where-Object { $_.Message -match '<demo-tenant-guid>' } |
  Select-Object TimeCreated, Id, Message | Format-List
```

The payload spells the whole problem out:

```
AADSTS90072: User account '{EUII Hidden}' from identity provider
'https://sts.windows.net/<corp-tenant>/' does not exist in tenant '<Demo>' and cannot access
the application '<guid>'(Dataverse CLI) in that tenant.
```

> **Filter on the tenant GUID, not on `AADSTS`.** Windows logs a constant background hum of
> `AADSTS50011` redirect-URI failures from ordinary first-party apps (Teams, OneDrive, Office).
> Those are noise. Only the events naming your demo tenant matter — in the verified case that was
> **8 events in 6 hours** out of hundreds.

**2. `dsregcmd /listaccounts` proves why it can never succeed silently:**

```
Account: u:…<corp-tenant-guid>, user: you@corp.com, authority: …/<corp-tenant-guid>
Accounts found: 1.
```

**One** account, in the corporate tenant. Nothing for the demo tenant. So every demo-tenant token
request falls back to the corporate identity, hits `AADSTS90072`, and escalates to an interactive
prompt.

**3. Correlate spawn → failure.** Compare process creation times with the AAD failures; they land
about one second apart, which confirms causation rather than coincidence:

```powershell
Get-CimInstance Win32_Process -Filter "Name='node.exe'" |
  Select-Object ProcessId, CreationDate,
    @{n='Org';e={ if($_.CommandLine -match 'mcp (https://[^/ ]+)'){$Matches[1]} }}
```

---

## ✅ Wiring a Dataverse MCP server to a demo tenant, in the right order

If you want Scout's `dataverse-*` tools pointed at your demo org, **the order is the whole trick.**
Registering the server before the CLI has an auth profile is what causes the endless Windows
sign-in prompts documented above.

### Step 1 — create the auth profile FIRST

```powershell
$dv = "$env:APPDATA\npm\node_modules\@microsoft\dataverse\bin\dataverse.js"
node $dv auth create --environment https://<your-org>.crm.dynamics.com/ --name <demo>
```

Sign in as the **demo tenant admin** (`admin@<your-tenant>.onmicrosoft.com`) — **not** your
corporate account. This is the same identity rule as the rest of the playbook, and it is where this
goes wrong: the browser may already be signed in as corp, so the account picker will offer the wrong
one by default.

Useful flags on `auth create`:

| Flag | When |
|---|---|
| `--deviceCode` / `-dc` | Interactive browser flow is blocked, or you are on a headless/remote box |
| `--username` / `--password` | Scripted setup — avoid; prefer interactive so no password is stored |
| `--applicationId` + `--clientSecret` + `--tenant` | Service-principal auth for an unattended pipeline |
| `--no-default-environment` | Do not make this the CLI's default target (sensible when juggling several demo orgs) |

Verify before going further — the profile must exist and point at the right URL:

```powershell
node $dv auth list
```

### Step 2 — register the MCP server SECOND

Add to `~/.scout/m-mcp-servers.json` under `servers` (back the file up first):

```json
"dataverse-<demo>": {
  "builtin": false,
  "config": {
    "name": "dataverse-<demo>",
    "type": "command",
    "command": "C:\\Program Files\\nodejs\\node.exe",
    "args": ["C:\\Users\\<you>\\AppData\\Roaming\\npm\\node_modules\\@microsoft\\dataverse\\bin\\dataverse.js",
             "mcp", "https://<your-org>.crm.dynamics.com/"]
  }
}
```

### Step 3 — restart the host and verify at the TOOL LIST

Restart is required; a running session has already spawned the old configuration.

**Do not trust the settings screen.** A server can read *registered* and *enabled* and still be
dead. The only proof is that its `dataverse-<demo>-*` tools actually appear in the agent's tool
list. Cross-check the host log:

```powershell
Get-Content "$env:USERPROFILE\.scout\m-diagnostics.jsonl" -Tail 200 |
  Select-String 'failed to connect'
```

Silence there plus visible tools = genuinely working.

### Maintenance: profiles expire, and the failure looks different

CDX demo tenants expire and their credentials rotate, so **`auth create` is not one-and-done.**

| Symptom | Meaning |
|---|---|
| No row in `auth list` for the org | Profile **never existed** — the endless-prompt case |
| Row exists, but the server still logs `failed to connect` | Profile exists and its **token has expired** — re-run `auth create` for that org |

On the verified machine `dataverse-cpg` was in exactly the second state: profile present, server
still failing. A registered-but-broken server is silent dead weight — it costs a spawn on every
session and contributes nothing.

> **Rule of thumb for a demo build:** do not register an MCP server for a tenant you are only going
> to touch through the browser. The Web API path needs no profile, no registration and no
> maintenance, and it is what every recipe in this skill actually uses.
