Files
parcer/README.md

161 lines
7.0 KiB
Markdown
Raw Normal View History

<h2 align="center">
2024-10-30 08:35:10 +05:30
<div>
2025-01-18 20:32:59 +05:30
<a href="https://www.maxun.dev/?ref=ghread">
<img src="/src/assets/maxunlogo.png" width="70" />
2024-10-30 08:35:10 +05:30
<br>
Maxun
</a>
</div>
2025-11-21 01:08:21 +05:30
Transform the Web into Structured Intelligence<br>
</h2>
2024-10-30 08:35:10 +05:30
<p align="center">
2025-11-24 21:06:44 +05:30
✨ Turn any website into clean, contextualized data pipelines for your AI applications ✨
2025-11-21 01:08:21 +05:30
<br />
2026-01-03 20:22:50 +05:30
Maxun is the easiest way to extract web data with no code.
2024-10-30 08:35:10 +05:30
</p>
<p align="center">
<a href="https://app.maxun.dev/?ref=ghread"><b>Go To App</b></a> •
<a href="https://docs.maxun.dev/?ref=ghread"><b>Documentation</b></a> •
<a href="https://www.maxun.dev/?ref=ghread"><b>Website</b></a> •
<a href="https://discord.gg/5GbPjBUkws"><b>Discord</b></a> •
2024-12-25 05:42:36 +05:30
<a href="https://www.youtube.com/@MaxunOSS?ref=ghread"><b>Watch Tutorials</b></a>
2024-11-14 07:32:43 +05:30
<br />
<br />
<a href="https://trendshift.io/repositories/12113" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12113" alt="getmaxun%2Fmaxun | Trendshift" style="width: 250px; height: 55px; margin-top: 10px;" width="250" height="55"/></a>
2024-10-30 08:35:10 +05:30
</p>
2025-11-21 01:08:21 +05:30
## What is Maxun?
2025-12-12 05:11:21 +05:30
Maxun helps you transform websites into structured APIs, clean markdown for AI workflows, and production-ready data pipelines — all in minutes.
2025-12-13 17:59:55 +05:30
### Ecosystem
2025-12-12 05:11:21 +05:30
2025-12-13 17:59:55 +05:30
1. **[Extract](https://docs.maxun.dev/category/extract)** Emulate real user behavior and collect structured data from any website. No code required.
* **[Recorder Mode](https://docs.maxun.dev/robot/extract/robot-actions)** - Record your actions as you browse; Maxun turns them into a reusable extraction robot.
* **[AI Mode](https://docs.maxun.dev/robot/extract/llm-extraction)** - Describe what you want in natural language and let LLM-powered extraction do the rest.
2. **[Scrape](https://docs.maxun.dev/robot/scrape/scrape-robots)** Convert full webpages into clean Markdown or HTML and capture screenshots. Ideal for AI workflows, agents, and document processing. No code required.
3. **[SDK](https://docs.maxun.dev/sdk/sdk-overview)** A complete developer toolkit for scraping, extraction, scheduling, and end-to-end data automation.
Whether you prefer browsing through a website or integrating automation into your codebase, Maxun adapts to your workflow.
2025-12-12 05:11:21 +05:30
2025-11-21 01:08:21 +05:30
2025-11-21 01:09:32 +05:30
## How Does It Work?
2025-11-21 01:08:21 +05:30
Maxun uses web robots to power everything you can do on the platform. There are two types of robots, each designed for a different job.
### 1. Extract Robots
2025-12-12 05:11:21 +05:30
**Extract robots emulate real user behavior and capture structured data.**
Choose how to build them
2025-12-13 17:59:55 +05:30
### a. Recorder Mode: Record your actions as you browse
2025-12-12 05:11:21 +05:30
- Build robots visually by browsing like a human.
- Perfect for structured, deterministic data extraction.
### Example: Extract 10 Property Listings from Airbnb
[https://github.com/user-attachments/assets/recorder-mode-demo-video](https://github.com/user-attachments/assets/c6baa75f-b950-482c-8d26-8a8b6c5382c3)
### b. LLM Extraction (Beta): Describe what you want in plain language
- Use natural language to define extraction patterns.
- Works with closed source & open source LLMs.
Get Started with LLM Extraction: https://docs.maxun.dev/robot/extract/llm-extraction
### Example: Extract Names, Rating & Duration of Top 50 Movies from IMDb
https://github.com/user-attachments/assets/f714e860-58d6-44ed-bbcd-c9374b629384
### Core capabilities
2025-11-21 01:08:21 +05:30
- Extract from any website, including behind logins
- Convert sites into APIs, spreadsheets, and workflows
- Scale extractions and run on schedules or via API
- Handle infinite scrolling and pagination
- Auto-adapt to website layout & structural changes
### 2. Scrape Robots
**Built for clean content and AI workflows.**
- Get clean HTML and LLM-ready Markdown from any website
- Remove scripts, styling, ads, and clutter automatically
- Perfect for RAG systems, AI summarization, embeddings, and content pipelines
2025-12-12 05:11:21 +05:30
- Ideal for feeding clean data to LLMs
### Example: Scrape GitHub Trending Repositories in clean Markdown format
2025-11-21 01:08:21 +05:30
https://github.com/user-attachments/assets/c774cbd4-5a85-45b7-b41f-128ee570eae6
## Quick Start
### Getting Started
2025-11-21 01:08:21 +05:30
The simplest & fastest way to get started is to use the hosted version: https://app.maxun.dev. You can self-host if you prefer!
### Installation
2025-12-12 05:11:21 +05:30
Maxun can run locally with or without Docker
1. [Setup with Docker Compose](https://docs.maxun.dev/installation/docker)
2. [Setup without Docker](https://docs.maxun.dev/installation/local)
3. [Environment Variables](https://docs.maxun.dev/installation/environment_variables)
2025-12-12 05:11:21 +05:30
4. [SDK](https://github.com/getmaxun/node-sdk)
2025-11-21 01:08:21 +05:30
### Upgrading & Self Hosting
1. [Self Host Maxun With Docker & Portainer](https://docs.maxun.dev/self-host)
2. [Upgrade Maxun With Docker Compose Setup](https://docs.maxun.dev/installation/upgrade#upgrading-with-docker-compose)
3. [Upgrade Maxun Without Docker Compose Setup](https://docs.maxun.dev/installation/upgrade#upgrading-with-local-setup)
2025-11-21 01:08:21 +05:30
## Sponsors
2024-10-30 09:06:36 +05:30
<table>
<tr>
2025-10-30 16:14:35 +05:30
<td width="229">
<br/>
<a href="https://www.lambdatest.com/?utm_source=maxun&utm_medium=sponsor" target="_blank">
<img src="https://github.com/user-attachments/assets/904dd40e-0498-47dd-98f1-7fa6d318adb9" /><br/><br/>
<b>LambdaTest</b>
</a>
<br/>
<sub>GenAI-powered Quality Engineering Platform that empowers teams to test intelligently, smarter, and ship faster.</sub>
</td>
</tr>
</table>
2025-11-21 01:08:21 +05:30
## Features
2025-12-13 17:59:55 +05:30
-**Extract Data With No-Code** Point and click interface
-**LLM-Powered Extraction** Describe what you want; use LLMs to scrape structured data
-**Developer SDK** Programmatic extraction, scheduling, and robot management
-**Handle Pagination & Scrolling** Automatic navigation
-**Run Robots On Schedules** Set it and forget it
-**Turn Websites to APIs** RESTful endpoints from any site
-**Turn Websites to Spreadsheets** Direct data export to Google Sheets & Airtable
-**Adapt To Website Layout Changes** Auto-recovery from site updates
-**Extract Behind Login** Handle authentication seamlessly
-**Integrations** Connect with your favorite tools
-**MCP Support** Model Context Protocol integration
-**LLM-Ready Data** Clean Markdown for AI applications
-**Self-Hostable** Full control over your infrastructure
-**Open Source** Transparent and community-driven
## Use Cases
2025-10-03 18:54:37 +05:30
Maxun can be used for various use-cases, including lead generation, market research, content aggregation and more.
View use-cases in detail here: https://www.maxun.dev/#usecases
2024-10-30 08:35:10 +05:30
2025-12-13 17:59:55 +05:30
## Note
2025-06-26 20:12:39 +05:30
This project is in early stages of development. Your feedback is very important for us - we're actively working on improvements. </a>
2024-10-30 09:25:59 +05:30
2025-12-13 17:59:55 +05:30
## License
2024-10-30 08:35:10 +05:30
<p>
This project is licensed under <a href="./LICENSE">AGPLv3</a>.
</p>
2025-12-13 17:59:55 +05:30
## Support Us
2025-10-10 00:40:53 +05:30
Star the repository, contribute if you love what were building, or [sponsor us](https://github.com/sponsors/amhsirak).
2025-08-12 03:19:50 +05:30
2025-12-13 17:59:55 +05:30
## Contributors
2024-10-30 08:35:10 +05:30
Thank you to the combined efforts of everyone who contributes!
2024-10-30 12:19:29 +05:30
<a href="https://github.com/getmaxun/maxun/graphs/contributors">
<img src="https://contrib.rocks/image?repo=getmaxun/maxun" />
2024-10-30 08:35:10 +05:30
</a>