377 lines
12 KiB
Plaintext
377 lines
12 KiB
Plaintext
|
|
---
|
||
|
|
title: Troubleshooting Guide
|
||
|
|
subtitle: Common issues, step timeline, and when to adjust what
|
||
|
|
slug: debugging/troubleshooting-guide
|
||
|
|
---
|
||
|
|
|
||
|
|
When a run fails or produces unexpected results, this guide helps you identify the cause and fix it.
|
||
|
|
|
||
|
|
## Quick debugging checklist
|
||
|
|
|
||
|
|
<Frame>
|
||
|
|
<img src="/images/debugging-checklist.png" alt="Skyvern Debugging Checklist" />
|
||
|
|
</Frame>
|
||
|
|
|
||
|
|
<Steps>
|
||
|
|
<Step title="Check run status">
|
||
|
|
Call `get_run(run_id)` and check `status` and `failure_reason`
|
||
|
|
</Step>
|
||
|
|
<Step title="Find the failed step">
|
||
|
|
Call `get_run_timeline(run_id)` to see which block failed
|
||
|
|
</Step>
|
||
|
|
<Step title="Inspect artifacts">
|
||
|
|
Check `screenshot_final` to see where it stopped, `recording` to watch what happened
|
||
|
|
</Step>
|
||
|
|
<Step title="Determine the fix">
|
||
|
|
Adjust prompt, parameters, or file a bug based on what you find
|
||
|
|
</Step>
|
||
|
|
</Steps>
|
||
|
|
|
||
|
|
**Key artifacts to check:**
|
||
|
|
- `screenshot_final` — Final page state
|
||
|
|
- `recording` — Video of what happened
|
||
|
|
- `llm_response_parsed` — What the AI decided to do
|
||
|
|
- `visible_elements_tree` — What elements were detected
|
||
|
|
- `har` — Network requests if something failed to load
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Step 1: Check the run
|
||
|
|
|
||
|
|
<CodeGroup>
|
||
|
|
```python Python
|
||
|
|
import os
|
||
|
|
from skyvern import Skyvern
|
||
|
|
|
||
|
|
client = Skyvern(api_key=os.getenv("SKYVERN_API_KEY"))
|
||
|
|
|
||
|
|
run = await client.get_run(run_id)
|
||
|
|
|
||
|
|
print(f"Status: {run.status}")
|
||
|
|
print(f"Failure reason: {run.failure_reason}")
|
||
|
|
print(f"Step count: {run.step_count}")
|
||
|
|
print(f"Output: {run.output}")
|
||
|
|
```
|
||
|
|
|
||
|
|
```typescript TypeScript
|
||
|
|
import { SkyvernClient } from "@skyvern/client";
|
||
|
|
|
||
|
|
const client = new SkyvernClient({
|
||
|
|
apiKey: process.env.SKYVERN_API_KEY,
|
||
|
|
});
|
||
|
|
|
||
|
|
const run = await client.getRun(runId);
|
||
|
|
|
||
|
|
console.log(`Status: ${run.status}`);
|
||
|
|
console.log(`Failure reason: ${run.failure_reason}`);
|
||
|
|
console.log(`Step count: ${run.step_count}`);
|
||
|
|
console.log(`Output: ${JSON.stringify(run.output)}`);
|
||
|
|
```
|
||
|
|
|
||
|
|
```bash cURL
|
||
|
|
curl -X GET "https://api.skyvern.com/v1/runs/$RUN_ID" \
|
||
|
|
-H "x-api-key: $SKYVERN_API_KEY" \
|
||
|
|
| jq '{status, failure_reason, step_count, output}'
|
||
|
|
```
|
||
|
|
</CodeGroup>
|
||
|
|
|
||
|
|
| Status | What it means | Likely cause |
|
||
|
|
|--------|---------------|--------------|
|
||
|
|
| `completed` | Run finished, but output may be wrong | Prompt interpretation issue |
|
||
|
|
| `failed` | System error | Browser crash, network failure |
|
||
|
|
| `terminated` | AI gave up | Login blocked, CAPTCHA, page unavailable |
|
||
|
|
| `timed_out` | Exceeded `max_steps` | Task too complex, or AI got stuck |
|
||
|
|
| `canceled` | Manually stopped | — |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Step 2: Read the step timeline
|
||
|
|
|
||
|
|
The timeline shows each block and action in execution order.
|
||
|
|
|
||
|
|
<CodeGroup>
|
||
|
|
```python Python
|
||
|
|
timeline = await client.get_run_timeline(run_id)
|
||
|
|
|
||
|
|
for item in timeline:
|
||
|
|
if item.type == "block":
|
||
|
|
block = item.block
|
||
|
|
print(f"Block: {block.label}")
|
||
|
|
print(f" Status: {block.status}")
|
||
|
|
print(f" Duration: {block.duration}s")
|
||
|
|
if block.failure_reason:
|
||
|
|
print(f" Failure: {block.failure_reason}")
|
||
|
|
for action in block.actions:
|
||
|
|
print(f" Action: {action.action_type} -> {action.status}")
|
||
|
|
```
|
||
|
|
|
||
|
|
```typescript TypeScript
|
||
|
|
const timeline = await client.getRunTimeline(runId);
|
||
|
|
|
||
|
|
for (const item of timeline) {
|
||
|
|
if (item.type === "block") {
|
||
|
|
const block = item.block;
|
||
|
|
console.log(`Block: ${block.label}`);
|
||
|
|
console.log(` Status: ${block.status}`);
|
||
|
|
console.log(` Duration: ${block.duration}s`);
|
||
|
|
if (block.failure_reason) {
|
||
|
|
console.log(` Failure: ${block.failure_reason}`);
|
||
|
|
}
|
||
|
|
for (const action of block.actions || []) {
|
||
|
|
console.log(` Action: ${action.action_type} -> ${action.status}`);
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
```bash cURL
|
||
|
|
curl -X GET "https://api.skyvern.com/v1/runs/$RUN_ID/timeline" \
|
||
|
|
-H "x-api-key: $SKYVERN_API_KEY"
|
||
|
|
```
|
||
|
|
</CodeGroup>
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Step 3: Inspect artifacts
|
||
|
|
|
||
|
|
<CodeGroup>
|
||
|
|
```python Python
|
||
|
|
artifacts = await client.get_run_artifacts(
|
||
|
|
run_id,
|
||
|
|
artifact_type=["screenshot_final", "llm_response_parsed", "skyvern_log"]
|
||
|
|
)
|
||
|
|
|
||
|
|
for a in artifacts:
|
||
|
|
print(f"{a.artifact_type}: {a.signed_url}")
|
||
|
|
```
|
||
|
|
|
||
|
|
```typescript TypeScript
|
||
|
|
const artifacts = await client.getRunArtifacts(runId, {
|
||
|
|
artifact_type: ["screenshot_final", "llm_response_parsed", "skyvern_log"]
|
||
|
|
});
|
||
|
|
|
||
|
|
for (const a of artifacts) {
|
||
|
|
console.log(`${a.artifact_type}: ${a.signed_url}`);
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
```bash cURL
|
||
|
|
curl -X GET "https://api.skyvern.com/v1/runs/$RUN_ID/artifacts?artifact_type=screenshot_final&artifact_type=llm_response_parsed&artifact_type=skyvern_log" \
|
||
|
|
-H "x-api-key: $SKYVERN_API_KEY" \
|
||
|
|
| jq '.[] | "\(.artifact_type): \(.signed_url)"'
|
||
|
|
```
|
||
|
|
</CodeGroup>
|
||
|
|
|
||
|
|
See [Using Artifacts](/debugging/using-artifacts) for the full list of artifact types.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Common issues and fixes
|
||
|
|
|
||
|
|
### Run Status Issues
|
||
|
|
|
||
|
|
<Accordion title="Run completed but output is wrong">
|
||
|
|
**Symptom:** Status is `completed`, but the extracted data is incorrect or incomplete.
|
||
|
|
|
||
|
|
**Check:** `screenshot_final` (right page?) and `llm_response_parsed` (right elements?)
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Add specific descriptions in your schema: `"description": "The price in USD, without currency symbol"`
|
||
|
|
- Add visual hints: "The price is displayed in bold below the product title"
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="Run timed out">
|
||
|
|
**Symptom:** Status is `timed_out`.
|
||
|
|
|
||
|
|
**Check:** `recording` (stuck in a loop?) and `step_count` (how many steps?)
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Increase `max_steps` if the task genuinely needs more steps
|
||
|
|
- Add explicit completion criteria: "COMPLETE when you see 'Order confirmed'"
|
||
|
|
- Break complex tasks into multiple workflow blocks
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="Run stuck in queued state">
|
||
|
|
**Symptom:** Run stays in `queued` status and never starts.
|
||
|
|
|
||
|
|
**Likely causes:** Concurrency limit hit, sequential run lock, or high platform load.
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Wait for other runs to complete
|
||
|
|
- Check your plan's concurrency limits in Settings
|
||
|
|
- If using "Run Sequentially" with credentials, ensure prior runs finish
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="504 Gateway Timeout on browser session">
|
||
|
|
**Symptom:** Getting `504 Gateway Timeout` errors when creating browser sessions.
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Retry after a few seconds — usually transient
|
||
|
|
- For self-hosted: check infrastructure resources
|
||
|
|
- If persistent, contact support with your run IDs
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
### Element & Interaction Issues
|
||
|
|
|
||
|
|
<Accordion title="AI clicked the wrong element">
|
||
|
|
**Symptom:** Recording shows the AI clicking something other than intended.
|
||
|
|
|
||
|
|
**Check:** `visible_elements_tree` (element detected?) and `llm_prompt` (target described clearly?)
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Add distinguishing details: "Click the **blue** Submit button at the **bottom** of the form"
|
||
|
|
- Reference surrounding text: "Click Download in the row that says 'January 2024'"
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="Element not found">
|
||
|
|
**Symptom:** `failure_reason` mentions an element couldn't be found.
|
||
|
|
|
||
|
|
**Check:** `screenshot_final` (element visible?) and `visible_elements_tree` (element detected?)
|
||
|
|
|
||
|
|
**Likely causes:** Dynamic loading, iframe, below the fold, unusual rendering (canvas, shadow DOM)
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Add `max_screenshot_scrolls` to capture lazy-loaded content
|
||
|
|
- Describe elements visually rather than by technical properties
|
||
|
|
- Add wait or scroll instructions in your prompt
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="Date fields entering wrong values">
|
||
|
|
**Symptom:** Dates entered in wrong format (MM/DD vs DD/MM) or wrong date entirely.
|
||
|
|
|
||
|
|
**Check:** `recording` (how did AI interact with the date picker?)
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Specify exact format: "Enter the date as **15/01/2024** (DD/MM/YYYY format)"
|
||
|
|
- For date pickers: "Click the date field, then select January 15, 2024 from the calendar"
|
||
|
|
- If site auto-formats: "Type 01152024 and let the field format it"
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="Address autocomplete cycling or wrong selection">
|
||
|
|
**Symptom:** AI keeps clicking through address suggestions in a loop, or selects wrong address.
|
||
|
|
|
||
|
|
**Check:** `recording` (is autocomplete dropdown appearing?)
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Specify selection: "Type '123 Main St' and select the first suggestion showing 'New York, NY'"
|
||
|
|
- Disable autocomplete: "Type the full address without using autocomplete suggestions"
|
||
|
|
- Break into steps: "Enter street address, wait for dropdown, select the matching suggestion"
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
### Authentication Issues
|
||
|
|
|
||
|
|
<Accordion title="Login failed">
|
||
|
|
**Symptom:** Status is `terminated` with login-related failure reason.
|
||
|
|
|
||
|
|
**Check:** `screenshot_final` (error message?) and `recording` (fields filled correctly?)
|
||
|
|
|
||
|
|
**Likely causes:** Expired credentials, CAPTCHA, MFA required, automation detected
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Verify credentials in your credential store
|
||
|
|
- Use `RESIDENTIAL_ISP` proxy for more stable IPs
|
||
|
|
- Configure TOTP if MFA is required
|
||
|
|
- Use a browser profile with an existing session
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="CAPTCHA blocked the run">
|
||
|
|
**Symptom:** `failure_reason` mentions CAPTCHA.
|
||
|
|
|
||
|
|
**Fix options:**
|
||
|
|
1. **Browser profile** — Use an authenticated session that skips login
|
||
|
|
2. **Human interaction block** — Pause for manual CAPTCHA solving
|
||
|
|
3. **Different proxy** — Use `RESIDENTIAL_ISP` for static IPs sites trust more
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
### Data & Output Issues
|
||
|
|
|
||
|
|
<Accordion title="AI hallucinating or inventing data">
|
||
|
|
**Symptom:** Extracted data contains information not on the page.
|
||
|
|
|
||
|
|
**Check:** `screenshot_final` (data visible?) and `llm_response_parsed` (what did AI claim to see?)
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Add instruction: "Only extract data **visibly displayed** on the page. Return null if not found"
|
||
|
|
- Make schema fields optional where appropriate
|
||
|
|
- Add validation: "If price is not displayed, set price to null instead of guessing"
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="File downloads not working">
|
||
|
|
**Symptom:** Files download repeatedly, wrong files, or downloads don't complete.
|
||
|
|
|
||
|
|
**Check:** `recording` (download button clicked correctly?) and `downloaded_files` in response
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Be specific: "Click the **PDF** download button, not the Excel one"
|
||
|
|
- Add wait time: "Wait for the download to complete"
|
||
|
|
- For multi-step downloads: "Click Download, then select PDF format, then click Confirm"
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
### Environment Issues
|
||
|
|
|
||
|
|
<Accordion title="Site showing content in wrong language">
|
||
|
|
**Symptom:** Site displays in unexpected language based on proxy location.
|
||
|
|
|
||
|
|
**Check:** What `proxy_location` is being used? Does site geo-target?
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Use appropriate proxy: `RESIDENTIAL_US` for English US sites
|
||
|
|
- Add instruction: "If the site asks for language preference, select English"
|
||
|
|
- Set language in URL: `https://example.com/en/`
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
<Accordion title="Workflow gets stuck in a loop">
|
||
|
|
**Symptom:** Recording shows AI repeating the same actions without progressing.
|
||
|
|
|
||
|
|
**Check:** `recording` (what pattern is repeating?) and conditional block exit conditions
|
||
|
|
|
||
|
|
**Fix:**
|
||
|
|
- Add termination: "STOP if you've attempted this action 3 times without success"
|
||
|
|
- Add completion markers: "COMPLETE when you see 'Success' message"
|
||
|
|
- Review loop conditions — ensure exit criteria are achievable
|
||
|
|
</Accordion>
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## When to adjust prompts vs parameters
|
||
|
|
|
||
|
|
<CardGroup cols={3}>
|
||
|
|
<Card title="Adjust prompt" icon="pen">
|
||
|
|
- AI misunderstood what to do
|
||
|
|
- Clicked wrong element
|
||
|
|
- Ambiguous completion criteria
|
||
|
|
- Extracted wrong data
|
||
|
|
</Card>
|
||
|
|
<Card title="Adjust parameters" icon="sliders">
|
||
|
|
- Timed out → increase `max_steps`
|
||
|
|
- Site blocked → change `proxy_location`
|
||
|
|
- Content didn't load → increase `max_screenshot_scrolls`
|
||
|
|
</Card>
|
||
|
|
<Card title="File a bug" icon="bug">
|
||
|
|
- Visible element not detected
|
||
|
|
- Standard UI patterns don't work
|
||
|
|
- Consistent wrong element despite clear prompts
|
||
|
|
</Card>
|
||
|
|
</CardGroup>
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Next steps
|
||
|
|
|
||
|
|
<CardGroup cols={2}>
|
||
|
|
<Card
|
||
|
|
title="Using Artifacts"
|
||
|
|
icon="file-lines"
|
||
|
|
href="/debugging/using-artifacts"
|
||
|
|
>
|
||
|
|
Detailed reference for all artifact types
|
||
|
|
</Card>
|
||
|
|
<Card
|
||
|
|
title="Reliability Tips"
|
||
|
|
icon="shield-check"
|
||
|
|
href="/going-to-production/reliability-tips"
|
||
|
|
>
|
||
|
|
Write prompts that fail less often
|
||
|
|
</Card>
|
||
|
|
</CardGroup>
|