A Python-based price comparison tool that collects product listings from multiple e-commerce platforms and presents them in one place.
Comparing the price of a product across different e-commerce platforms usually means searching for the same product multiple times and manually comparing the results.
Many platforms also don't provide a public API that makes this kind of comparison straightforward, which makes collecting publicly displayed product information more complicated.
I wanted to turn that repetitive process into a single tool: search once, collect the available results, and compare them side by side.
The scraper accepts a product search query and retrieves matching listings and prices from supported e-commerce platforms.
Results from the different sources are then normalized and presented together so users can quickly compare available products and prices.
The project can be used in three different ways:
No platform-specific API key is required for the supported sources.
The application takes a product query and sends it through the individual source handlers responsible for retrieving and parsing results from each supported platform.
Because every e-commerce website structures its pages differently, each source requires its own extraction logic. The collected information is then converted into a common result format before being displayed to the user.
One of the goals of the project was to avoid tying the scraper to a single type of interface.
The desktop interface provides a more accessible way to run searches, while the command-line interface makes the tool useful for scripting and experimentation. The Flask web interface provides a browser-based alternative.
All three interfaces use the same underlying scraping logic, keeping the actual data collection separate from how users interact with the application.
The project is built in Python, with BeautifulSoup handling HTML parsing and Flask powering the web interface.
Some supported websites use techniques that make traditional HTTP-based scraping unreliable. For those cases, the project can use an automated browser through Undetected Chromedriver to render pages before extracting the relevant information.
The goal is to retrieve publicly displayed product information while keeping the scraping process practical across different website implementations.
The biggest challenge isn't sending a request to a website. It's dealing with the fact that websites rarely agree on how their information is structured.
Product names, prices, links, availability information, and page layouts can all differ between platforms. Those layouts can also change over time.
The parsing logic therefore needs to be defensive. If a particular element changes or isn't available, the application should handle the situation gracefully rather than causing the entire comparison process to fail.
This project started from a simple everyday problem: comparing prices shouldn't require opening a collection of browser tabs and doing the work manually.
It also gave me an opportunity to explore web scraping from a practical perspective - dealing with real websites, inconsistent markup, browser automation, multiple interfaces, and data normalization.
Building the scraper taught me that reliable web scraping is less about extracting a value once and more about designing for change.
Website structures evolve, different platforms expose different information, and automated browsers introduce another layer of complexity. Keeping the scraping logic modular and handling failures gracefully became just as important as successfully extracting the data.
The complete source code, setup instructions, supported platforms, and usage documentation are available on GitHub.
