About
Social media data crawler. Supports content scraping from Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Zhihu and more.
Our Verdict
Media Crawler is a valuable addition to any AI agent toolkit. Its open-source nature and active development make it a reliable choice for both personal and professional use.
Pricing
Use Cases
- Using Media Crawler with Claude, Cursor, and other AI agents
- Integrating Media Crawler into existing development workflows
- Automating developer tasks with AI assistance
Pros
- Media Crawler integrates seamlessly into development workflows
- Open source with active community support
- Regular updates and feature improvements
Cons
- Media Crawler requires some learning to use effectively
- Documentation could be more comprehensive
- May need additional tools for full workflow
Installation
See README for installation instructions.Frequently Asked Questions
Which platforms does Media Crawler support?
It covers major Chinese social media platforms: Xiaohongshu (RED), Douyin, Kuaishou, Bilibili, Weibo, Zhihu and more, scraping public content like posts, video metadata and comments — useful for content research, sentiment analysis and data insights.
How do I install Media Crawler?
It is a Python project: clone the GitHub repository, install dependencies, then configure target platforms and login state per the README. Paired with an AI agent, the AI can interpret scraped results and generate analysis reports.
Is using Media Crawler legal and compliant?
The tool is free and open source, intended for learning and research only. Scraping must respect each platform’s terms of service and local laws: throttle request rates, collect only public data, and never use it for commercial resale or privacy violations. Users bear the risk.