vodorod/Dorod-Sky
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Migrate docs to fern (#1652)

Browse Source
This commit is contained in:
Suchintan
2025-01-27 10:06:11 -05:00
committed by GitHub
parent 2100174a1b
commit bf2d9ef737
101 changed files with 4437 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

163
fern/running-tasks/advanced-features.mdx Normal file
View File

@@ -0,0 +1,163 @@
---
title: Advanced Settings for tasks
subtitle: How to use advanced settings for tasks
slug: running-tasks/advanced-features
---
## Overriding the max_steps paramter in the task
If you want the task to execute for a specific number of steps, you can use the `x-max-steps-override` header in the request. This will override the default `max_steps` parameter set in the task.
```json
POST https://api.skyvern.com/api/v1/tasks
Headers:
{
"x-api-key": "YOUR_API_KEY",
"x-max-steps-override": 10
}
Body:
.. usual task body ..
```
## Time-based One-time Password (TOTP)
Skyvern supports one-time password (see https://www.twilio.com/docs/glossary/totp for more information), also known as 2FA/MFA, in two ways: 1) Skyvern gets the code from your endpoint 2) You push the the code to Skyvern.
### Get Code From Your Endpoint
You can pass `totp_verification_url` when [creating a task](/running-tasks/api-spec#request-initiate-a-task). Inside this endpoint hosted by you, you have to conform to the following schema:
#### Set Up Your TOTP Endpoint
For websites that requires a verification code to complete a task, you have to set up a TOTP endpoint for Skyvern to fetch the verification code.
Here's the TOTP endpoint contract you should use:
Request (POST):
| Parameter | Type | Required? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| task_id | String | yes | tsk_123 | The task ID that needs the verification to be done |
Response:
| Parameter | Type | Required? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| task_id | String | yes | tsk_123 | The task ID that needs the verification to be done |
| verification_code | String | no | 123456 | The verification code |
#### Validate The Sender of The Request
Same as the webhook API, your server needs to make sure it’s Skyvern that’s making the request.
- a python example for how to generate and validate the signature:
```python
def validate_skyvern_request_headers(request: Request) -> bool:
header_skyvern_signature = request.headers["x-skyvern-signature"]
payload = request.body() # this is a bytes
hash_obj = hmac.new(SKYVERN_API_KEY.encode("utf-8"), msg=payload, digestmod=hashlib.sha256)
client_generated_signature = hash_obj.hexdigest()
return header_skyvern_signature == client_generated_signature
```
SKYVERN_API_KEY: this is the [api key](/running-tasks/introduction) specific to your organization
### Push Code To Skyvern
You can pass `totp_identifier` when [creating a task](/running-tasks/api-spec#request-initiate-a-task). When the TOTP code arrives at your inbox or as a text message, all you need to do is to send the email/message (Gmail + Zapier integration can be a good solution to set up email forwarding) to Skyvern's TOTP receiver endpoint (see below)
#### Skyvern's TOTP Endpoint
This endpoint takes your TOTP/2FA/MFA code, stores it in Skyvern’s database and uses it while running tasks/workflows.
Request type: `POST`
Endpoint url: `https://api.skyvern.com/api/v1/totp`
Authentication: same as other Skyvern APIs, you need to pass your api key with the `x-api-key` header.
Request data:
| field | required | type | example | description |
| --- | --- | --- | --- | --- |
| totp_identifier | yes | string | An email address or phone number which received the code | this is a required field as this is the best way for skyvern to know what the identifier |
| content | yes | string | | the email content of a 2FA email; the text message for the verification code |
| task_id | no | string | tsk_22222222 | used to help skyvern locate the totp code more accurately for your task |
| workflow_id | no | string | wpid_12345 | used to help better locate the totp code accurately for your workflow |
| source | no | string | email, phone, app, etc | |
| expired_at | no | timestamp | 2024-09-18T05:15:02 | The expiration time of the code. If provided, skyvern uses it to decide if the code is valid |
#### Forwarding Your Email To Skyvern (Gmail + Zapier)
This setup requires a Zapier pro plan account.
**Step 1. Create a Zapier Zap**
Go to https://zapier.com/app/home and create new Zaps
<p align="center">
<img src="../images/totp/create_zap.png"/>
</p>
In the newly created Zap draft, Click the “Trigger” button
<p align="center">
<img src="../images/totp/create_zap_trigger.png"/>
</p>
Click `Email by Zapier`
<p align="center">
<img src="../images/totp/create_zap_trigger.png"/>
</p>
In the Email “Setup”, pick `New Inbound Email` in the `Trigger event` selection. Click `Continue` to complete the “Setup”
<p align="center">
<img src="../images/totp/create_zap_email_setup.png"/>
</p>
In Email “Configure”, create an email address which will be used to forward emails for TOTP codes. Click “Continue”.
<p align="center">
<img src="../images/totp/create_zap_input_email.png"/>
</p>
Let’s add the Action to complete the Zapier setup before coming back to test it. Click the “Action” button and add `Webhooks by Zapier`
<p align="center">
<img src="../images/totp/create_zap_action.png"/>
</p>
<p align="center">
<img src="../images/totp/create_zap_webhook.png"/>
</p>
In the Setup, choose “POST” under the `Action event` selection. Then click “Continue”.
<p align="center">
<img src="../images/totp/create_zap_action_event_post.png"/>
</p>
In the “Configure”, set up these in order to make a POST request to Skyvern’s TOTP API:
- URL: [`https://api.skyvern.com/api/v1/totp`](https://api.skyvern.com/api/v1/totp)
- Payload Type: json
- Data:
- totp_identifier: choose `Raw To Email` after clicking the “+” sign
- content: choose `Body Plain` after clicking the “+” sign
- source: email
- Headers:
- x-api-key: `Your Skyvern API Key`
<p align="center">
<img src="../images/totp/create_zap_webhook_complete.png"/>
</p>
Click Continue
**Step 2. Add forwarding email and create a filter in Gmail**
Go to Gmail Settings → Forwarding and POP/IMAP (https://mail.google.com/mail/u/0/#settings/fwdandpop) → click “Add a forwarding address” → enter the zapier email address you just created. There might be some verifications, including a verification email from Zapier, you have to complete here.
After setting up the forwarding email address, go to “Filters and Blocked Addresses” (https://mail.google.com/mail/u/0/#settings/filters). Click “Create a new filter” and set up your email filtering rule for your TOTP (2FA/MFA) emails. Click “Create filter”. Check “Forward it to” and pick the new email address and update filter.
<p align="center">
<img src="../images/totp/create_email_forwarding.png"/>
</p>
**Step 3. Test it end to end!**
You can forward any previous TOTP (2FA/MFA) email to the Zapier email address you created in Step 1.
In Zapier: under the “Test” of the Webhooks action, send a request to test it out. If your test is successful, you should see a `A request was sent to Webhooks by Zapier` message
<p align="center">
<img src="../images/totp/test_end_to_end.png"/>
</p>

151
fern/running-tasks/api-spec.mdx Normal file
View File

@@ -0,0 +1,151 @@
---
title: Tasks API
subtitle: The core building block in Skyvern
slug: running-tasks/api-spec
---
Tasks are the building block of Skyvern. They represent a single instruction (goal) to the browser to go do something using language models. Ex. “Go to alibaba and extract this information”
## Request - Initiate a task
Request type: `POST`
Production:`https://api.skyvern.com/api/v1/tasks/`
### Header
| Parameter | Type | Required? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| x-api-key | String | yes | [your-api-key-here] | Bearer token that gives your backend access to the Skyvern API. This will be manually provided by us |
| x-max-steps-override | Integer | no | 10 | Overrides any max step configuration for the initiated task |
### Body
| Parameter | Type | Required? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| url | HttpUrl | yes | https://www.example.com | It must be a http or https URL |
| navigation_goal | String | no | Apply for a job | The prompt that tells the agent what the user-facing goal is. This is the guiding light for the LLM as it navigates a particular website / sitemap to achieve this specified goal |
| data_extraction_goal | String | no | Was the job application successful? | The prompt that instructs the agent to extract information once the agent has achieved its user_goal |
| navigation_payload | Object | no | "name": "Chris P. Bacon",<br/>"email": "mailto:chris@pbacon.com" | JSON-formatted payload with any “facts” or information that would help the agent perform its job. In the case of navigating an insurance quote, this payload would include any user information to help fill out the insurance flow such as date of birth, or age they got their license, and so on<br/><br/>This can include nested information, and the formatting isn’t validated |
| webhook_callback_url | HttpUrl | no | … | The callback URL once our system is finished processing this async task |
| proxy_location | String | yes | RESIDENTIAL | Proxy location for the web-browsing request. Please pass RESIDENTIAL as a value |
| extracted_information_schema | JSON | no | | Used to enforce a JSON schema spec to be enforced in the data_extraction_goal. Similar to https://json-schema.org/ definition. |
| totp_verification_url | HttpUrl | no | https://mywebsite.com/two_factor_code | The url of your TOTP endpoint. If this field is provided, Skyvern will call the url to fetch the TOTP/2FA/MFA code when needed |
| totp_identifier | String | no | myemail@example.com / 4155558888 | The email address or the phone number which receives the TOTP/2FA/MFA code. If this field is provided, Skyvern will fetch the code that is pushed to [Skyvern's TOTP API](https://docs.skyvern.com/running-tasks/advanced-features#push-code-to-skyvern) |
## Example Request (Apply for a job)
```python
POST https://api.skyvern.com/api/v1/tasks/
{
"url": "https://jobs.lever.co/leverdemo-8/45d39614-464a-4b62-a5cd-8683ce4fb80a/apply",
"navigation_goal": "Apply for a job",
"data_extraction_goal": "Was the job application successful?",
"proxy_location": "RESIDENTIAL",
"navigation_payload": {
"name": "Chris P. Bacon",
"email": "chris@pbacon.com"
}
}
```
## Response
Each task has an associated `task_id` -- a unique identifier you can use to look up information about any task.
| Parameter | Type | Always returned? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| task_id | String | yes | t_123456 | The task id associated with this specific task |
## Response Webhook - Task conclusion (POST)
If a `webhook_callback_url` is specified within your task request, Skyvern will make a callback to your system letting you know that it has either finished, terminated or failed a task execution.
The following headers can be used to validate it's an authentic Skyvern request.
### Headers
| Parameter | Type | Required? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| x-skyvern-signature | String | yes | v0=a2114d57b48eac39b9ad189<br/>dd8316235a7b4a8d21a10bd275<br/>19666489c69b503 | Authentication token that allows our service to communicate with your backend service via callback / webhook <br/>We’ll be using the same strategy slack uses, as defined here: https://api.slack.com/authentication/verifying-requests-from-slack#making__validating-a-request |
| x-skyvern-timestamp | String | yes | 1531420618 | Timestamp used to decode and validate the incoming webhook call<br/><br/>We’ll be using the same strategy slack uses, as defined here: https://api.slack.com/authentication/verifying-requests-from-slack#making__validating-a-request |
### Body
These parameters are returned in the body of the `webhook_callback_url`.
| Parameter | Type | Always returned? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| task_id | String | yes | t_123456 | The task id associated with this specific task |
| status | String | yes | success | The status of the task |
| extracted_information | Object | Yes | 'price’: ‘$100.0’ | Unstructured JSON payload containing the extracted information as specified by the users’ input prompt |
| screenshot_url | String | Yes | … url to screenshot … | Screenshot of the final page, where the data extraction occurs |
| recording_url | String | Yes | .. url to recording … | Recording of the entire browsing session to help debug any issues |
| failure_reason | String | No | “Failed to pass this page - missing information: date of birth” | A failure reason that caused the job to fail. This can range from internal errors (Skyvern side) or external errors (insufficient information provided) |
## Request - Task Details (GET)
You can use this API to poll for task status updates if you don’t want to wait for webhook callbacks.
Request type: `GET`
Production:`api.skyvern.com/api/v1/tasks/{task_id}`
| Parameter | Type | Required? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| task_id | String | yes | t_123 | The id of the task you want to check the status of |
## Request - List Task Details (GET)
Request type: `GET`
Production:`api.skyvern.com/api/v1/tasks/`
| Parameter | Type | Required? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| page | Integer | no | 1 | default=1
has to be ≥1 |
| page_size | Integer | no | 10 | default=10
has to be ≥1 |
## Response - Task Details (GET)
| Parameter | Type | Sample Value | Description |
| --- | --- | --- | --- |
| request | JSON | | Includes the initial request sent to create the task. Fields included: url,webhook_callback_url,navigation_goal,data_extraction_goal,navigation_payload,proxy_location,extracted_information_schema |
| task_id | String | tsk_123 | The id of the task you want to check the status of |
| status | String | created / queued / running / completed / terminated / failed / canceled | String indicating the status of this task. created: the initial state when the task is created; queued: the task has been enqueued and is waiting to be picked up by a worker to execute; running: the task is being executed; completed: the task has successfully completed; terminated: the agent doesn't believe the task could be completed so it terminated the task; failed: the task failed due to a specific reason; canceled: the task is canceled by the user |
| created_at | Timestamp | 2022-11-22T22:55:55 | Timestamp indicating when the task was created. |
| modified_at | Timestamp | 2022-11-22T22:55:55 | Timestamp indicating when the task was last updated. Used to detect long-running tasks. |
| extracted_information | Object | 'price’: ‘$100.0’ | |
| screenshot_url | String | "https://skyvern-artifacts.s3.amazonaws.com/production/tsk_123/...screenshot_final.png" | … url to screenshot … |
| recording_url | String | "https://skyvern-artifacts.s3.amazonaws.com/production/tsk_123/...recording.webm" | ... url to recording … |
| action_screenshot_urls | List[str] | ["https://skyvern-artifacts.s3.amazonaws.com/production/tsk_123/step_2...action.png", "https://skyvern-artifacts.s3.amazonaws.com/production/tsk_123/step_0...action.png", "https://skyvern-artifacts.s3.amazonaws.com/production/tsk_123/step_0...action.png"] | 3 screenshots for the lastest 3 actions that Skyvern took. |
| failure_reason | String | “Failed to pass this page - missing information: invalid password” | The reason why the task failed. Examples: "Max steps per task (10) exceeded ..." - by default, a task as a cap on the number of steps per task for budget control. This can be configured in your organization settings page; "Max retries per step (3) exceeded..." - this means Skyvern ran into some internal failures. Skyvern Agent is not perfect and doesn't have 100% web coverage today. When it runs into pages it cannot navigate through or take actions on, this failure_reason shows up. (Currently we're improving Skyvern's abilility as well as the failure_reason to show better errors to help generate better feedbacks to your failure) |
## Request - List Steps (GET)
Each task is made up of "steps" which are the individual actions Skyvern takes to complete the task. You can use this endpoint to get all the steps of the task.
Request type: `GET`
Production: `https://api.skyvern.com/api/v1/tasks/{task_id}/steps`
## Response - List Steps (GET)
The response is a list of Step Object.
### Step Object
| Parameter | Type | Sample Value | Description |
| --- | --- | --- | --- |
| organization_id | String | o_123 | Your organization id |
| task_id | String | tsk_123 | The id of the task |
| step_id | String | stp_123 | The id of the step |
| status | String | created / running / completed / failed / canceled | The status of the step |
| order | Integer | 0 / 1 / 2 | The zero based index of the step. 0 is the first step of the task. |
| retry_index | Integer | 0 / 1 / 2 / 3 | When a step fails, the retry step will have a retry_index that's larger than 0 |
| input_token_count | Integer | 19223 | The number of input tokens used in this step |
| output_token_count | Integer | 500 | The number of output tokens generated in this step |
## Request - Cancel A Task (POST)
A task that's in any of thsese states can be canceled: ["created", "queued", "running"]. This stops the execution of a task.
Request type: `POST`
Production: `https://api.skyvern.com/api/v1/tasks/{task_id}/cancel`

16
fern/running-tasks/introduction.mdx Normal file
View File

@@ -0,0 +1,16 @@
---
title: Introduction
subtitle: Tasks are the building block of Skyvern
slug: running-tasks/introduction
---
# Getting your API Key
1. Navigate to [app.skyvern.com](https://app.skyvern.com) and log into your account
1. Validate that you're in the correct account (personal vs organization)
1. Click on "Settings"
1. Click Reveal to see your API key. This is the unique key associated with your account that allows you to intract with Skyvern
<img src="../images/api_key_image.png" />

58
fern/running-tasks/visualizing-results.mdx Normal file
View File

@@ -0,0 +1,58 @@
---
title: Visualizing Results
subtitle: How to understand the information shown by Skyvern
slug: running-tasks/visualizing-results
---
# Visualizing Results
Skyvern comes bundled with a Visualizer to help you understand what's going on with your tasks. To get started, navigate to the task history page and click on any task.
<img src="../images/visualizing_results/navigate_to_task.png" />
## Actions
Each action that Skyvern takes can be viewed in the Action viewer. It's accompanied with a screenshot of the screen state *after* the action has executed
<img src="../images/visualizing_results/view_actions.png" />
## Recording
Each Skyvern task comes with a recording of the entire operation (end to end). To view the recording, click on the recording tab.
<img src="../images/visualizing_results/view_recording.png" />
## Task Parameters
The task parameters are the inputs that you provided to Skyvern. This includes the URL, the extraction rules, and any other information that you provided to Skyvern.
<img src="../images/visualizing_results/view_parameters.png" />
## Diagnostic Logs
The diagnostics tab contains information used by Skyvern to do its processing. It includes information such as the annotated screenshots, action screenshots, element tree, prompt, action list, page HTML, and the raw LLM Request.
<img src="../images/visualizing_results/view_diagnostics.png" />
## Accessing results via API
You can also access the results of a task via the API.
The following endpoint can be used to get the results of a task:
```
GET https://api.skyvern.com/api/v1/tasks/{task_id}/steps?page=1&page_size=15
```
The response will contain a list of steps, alongside their actions and other relevant information
The following endpoint can be used to retrieve artifacts for a specific step:
```
GET https://api.skyvern.com/api/v1/tasks/{task_id}/steps/{step_id}/artifacts
```
## Artifacts configurations
By default, Skyvern stores artifacts, including video recording, screenshots, llm requests and responses, html and skyvern parsed html elements locally in the `/artifacts` folder under the skyvern repository.
You can also have skyvern to upload atrifacts to your s3 buckets. To do this, first set up these environment variables:
- `AWS_DEFAULT_REGION`: `us-east-1`, `us-west-1`, ...
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `SKYVERN_STORAGE_TYPE`: set it to be `s3`. The default is `local`
Make sure these s3 buckets are created: `skyvern-artifacts`, `skyvern-screenshots`. These are the default bucket names skyvern uses. To customize the bucket names, change these two env variables: `AWS_S3_BUCKET_ARTIFACTS` and `AWS_S3_BUCKET_SCREENSHOTS`

49
fern/running-tasks/webhooks-faq.mdx Normal file
View File

@@ -0,0 +1,49 @@
---
title: Webhooks FAQ
subtitle: How Skyvern notifies you when its done
slug: running-tasks/webhooks-faq
---
# FAQ
## Webhooks vs HTTP requests?
We use Webhooks for executing tasks as the expected runtime of these jobs can exceed default HTTP timeouts (1 minute)
## How do we handle webhook authentication? (ie how can we handle callbacks?)
<CodeGroup>
```python Python
import hmac
from fastapi import Request
def validate_skyvern_request_headers(request: Request) -> bool:
header_skyvern_signature = request.headers["x-skyvern-signature"]
payload = request.body() # this is a bytes
hash_obj = hmac.new(SKYVERN_API_KEY.encode("utf-8"), msg=payload, digestmod=hashlib.sha256)
client_generated_signature = hash_obj.hexdigest()
return header_skyvern_signature == client_generated_signature
```
```javascript Javascript
const crypto = require('crypto');
function validateSkyvernRequestHeaders(req) {
const headerSkyvernSignature = req.headers['x-skyvern-signature'];
const payload = req.body; // assuming req.body is a Buffer or string
const hash = crypto.createHmac('sha256', process.env.SKYVERN_API_KEY)
.update(payload)
.digest('hex');
return headerSkyvernSignature === hash;
}
```
</CodeGroup>
SKYVERN_API_KEY: this is the [api key](/running-tasks/introduction) specific to your organization
# Webhook common parameters
| Parameter | Type | Required? | Sample Value | Description |
| --- | --- | --- | --- | --- |
| webhook_callback_url | String | yes | … | |