X Scraper AI
AI-powered social media scraper
- Status
- Shipped
- Role
- Creator
- Source
- Open source
Overview
A data extraction and analytics service for Twitter/X. It pulls profiles and timelines, normalizes the result, and returns engagement metrics and profile analytics alongside the raw records.
The output shape is the point: clean, consistent records that go straight into a model prompt or a dataframe without a cleanup pass.
The problem
Raw social data arrives dense, deeply nested, and inconsistent between endpoints. Every downstream consumer ends up writing the same flattening and de-duplication code.
Analytics on top of it then get rebuilt per project, so engagement rates and profile summaries are computed slightly differently every time.
What was built
- A FastAPI service exposing extraction endpoints with a single normalized record schema across profiles and timelines.
- Engagement metric computation and profile analytics calculated once, in the service, rather than per consumer.
- Clean data processing tuned for AI integration — output is already flat and typed when it reaches a prompt.
Python
FastAPI