Next-gen analytics platform for Google Merchant Center Market Insights
Transform your e-commerce data into actionable insights with MerchVibeX - a powerful data pipeline and visualization dashboard that connects Google Merchant Center and Google Ads data to unlock pricing strategies, identify trending products, and optimize your competitive positioning.
MerchVibeX is a scalable data analytics solution that automates the extraction, processing, and visualization of Market Insights data from Google Merchant Center. Built on Google Cloud Platform, it empowers merchants to make data-driven decisions with real-time pricing benchmarks, best-selling product analysis, and competitive market intelligence.
- 📊 Price Benchmarking - Analyze competitive pricing data to optimize your product pricing strategies
- 🔥 Best Sellers Analysis - Identify trending products and brands to capitalize on market opportunities
- 💡 Price Suggestions - Get AI-powered recommendations for optimal sale pricing
- 📈 Inventory Insights - Spot high-demand products that are out of stock and driving potential sales
- ⚡ Automated Pipelines - Daily data synchronization with minimal manual intervention
- 🎯 Looker Studio Integration - Beautiful, interactive dashboards for seamless data visualization
- 🔍 Data Quality Monitoring - Built-in health checks and validation tools
- ✅ Configuration Validation - Pre-flight checks before deployment
┌─────────────────────────────────────────────────────────────────────────┐
│ MERCHVIBEX ARCHITECTURE │
└─────────────────────────────────────────────────────────────────────────┘
┌──────────────────────┐ ┌──────────────────────┐
│ Google Merchant │ │ Google Ads │
│ Center │ │ │
│ (Market Insights) │ │ (Shopping Campaigns)│
└──────────┬───────────┘ └──────────┬───────────┘
│ │
│ API Data Sync │ API Data Sync
│ (Automated Daily) │ (Automated Daily)
▼ ▼
┌──────────────────────────────────────────────────────────┐
│ BigQuery Data Transfer Service │
│ • Merchant Center Transfer │
│ • Google Ads Transfer │
│ • Scheduled Query Jobs │
└──────────────────────┬───────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────┐
│ Google BigQuery Data Warehouse │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Merchant │ │ Google Ads │ │ Reference │ │
│ │ Center Data │ │ Data │ │ Data │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└──────────────────────┬───────────────────────────────────┘
│
│ SQL Processing & Aggregation
▼
┌──────────────────────────────────────────────────────────┐
│ Data Processing Layer │
│ │
│ • Inventory View Procedures │
│ • Best Sellers Analysis Procedures │
│ • Price Competitiveness Calculations │
│ • Data Quality Validation │
└──────────────────────┬───────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────┐
│ Processed Views & Tables │
│ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ InventoryView│ │BestSeller │ │
│ │ │ │WeeklyProduct │ │
│ │• Price Data │ │View │ │
│ │• Competition │ │ │ │
│ │• Insights │ │• Trends │ │
│ └──────────────┘ │• Performance │ │
│ └──────────────┘ │
└──────────────────────┬───────────────────────────────────┘
│
│ Real-time Data Connection
▼
┌──────────────────────────────────────────────────────────┐
│ Looker Studio Dashboards │
│ │
│ 📊 Price Benchmark Dashboard │
│ 📈 Best Sellers Overview │
│ 💡 Price Suggestion Views │
│ 🎯 Inventory Insights │
└──────────────────────┬───────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────┐
│ Business Users & Teams │
│ │
│ • Pricing Teams • Merchandising • Marketing │
│ • Analytics • Business Intelligence │
└──────────────────────────────────────────────────────────┘
SETUP WORKFLOW:
┌─────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Validate │───▶│ Enable │───▶│ Create │───▶│ Deploy SQL │
│ Config │ │ Cloud APIs │ │ Transfers │ │ Procedures │
└─────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
│
▼
┌─────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Connect │◀───│ Schedule │◀───│ Initialize │◀───│ Load │
│ Dashboard │ │ Daily Jobs │ │ Dataset │ │ Reference │
└─────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
- Competitive Analysis: Compare your prices against market benchmarks
- Price Optimization: Make informed pricing decisions based on real market data
- Dynamic Pricing: Identify opportunities for price adjustments
- Trend Identification: Discover what's hot in your category
- Product Opportunities: Find gaps in your inventory for high-demand items
- Brand Performance: Track which brands are performing best
- Campaign Optimization: Align marketing strategies with trending products
- Inventory Alerts: Get notified about products that need restocking
- Performance Tracking: Monitor how pricing changes affect sales
- Python 3.8+ - Core automation and API integration
- Google BigQuery - Data warehouse and analytics engine
- BigQuery Data Transfer Service - Automated data synchronization
- Looker Studio - Data visualization and reporting
- Google Cloud Platform - Infrastructure and APIs
Before you begin, ensure you have:
- ✅ A Google Cloud Project with billing enabled
- ✅ Google Merchant Center account with Standard access
- ✅ Google Ads account with Standard access
- ✅ Editor/Owner role in your Google Cloud project
- ✅
gcloudCLI installed (or use Cloud Shell)
Before running the setup, validate your configuration:
python config_validator.py \
--project_id=<YOUR_PROJECT_ID> \
--merchant_id=<YOUR_MERCHANT_ID> \
--ads_customer_id=<YOUR_ADS_CUSTOMER_ID>-
Open Cloud Shell
# Navigate to Cloud Shell in Google Cloud Console -
Clone the Repository
git clone https://github.com/theyashdhiman04/MerchVibeX.git cd MerchVibeX -
Run Setup Script
sh setup.sh --project_id=<YOUR_PROJECT_ID> \ --merchant_id=<YOUR_MERCHANT_ID> \ --ads_customer_id=<YOUR_ADS_CUSTOMER_ID>
-
Install Google Cloud CLI
- Follow the official installation guide
-
Clone and Setup
git clone https://github.com/theyashdhiman04/MerchVibeX.git cd MerchVibeX sh setup.sh --project_id=<YOUR_PROJECT_ID> \ --merchant_id=<YOUR_MERCHANT_ID> \ --ads_customer_id=<YOUR_ADS_CUSTOMER_ID>
The setup script automates:
- ✅ Enables required Google Cloud APIs (BigQuery, Data Transfer)
- ✅ Creates automated data transfers for Merchant Center and Google Ads
- ✅ Sets up BigQuery datasets and tables
- ✅ Configures daily scheduled queries for data refresh
- ✅ Prepares data for Looker Studio visualization
Once your data pipeline is set up, connect it to Looker Studio:
-
Access the Template (if available)
- Join the dashboard template group or create your own
-
Connect Data Sources
- Navigate to
Resource > Manage added data sources - Connect to your BigQuery dataset:
merch_vibe_x - Link tables:
InventoryViewandBestSellerWeeklyProductView
- Navigate to
-
Customize & Share
- Customize the dashboard to your needs
- Share with your team for collaborative insights
Note: Performance metrics may take 12-24 hours to appear after initial setup.
MerchVibeX/
├── cloud_env_setup.py # Main setup orchestration
├── cloud_bigquery.py # BigQuery operations
├── cloud_data_transfer.py # Data transfer management
├── auth.py # Authentication utilities
├── setup.sh # Installation script
├── config_validator.py # Configuration validation
├── data_quality_checker.py # Data quality monitoring
├── monitoring.py # Transfer health monitoring
├── demo.py # Interactive demo script
├── sql/
│ ├── inventory.sql # Inventory view queries
│ ├── best_sellers.sql # Best sellers queries
│ └── main_workflow.sql # Daily workflow
├── plugins/
│ └── cloud_utils/ # Cloud API utilities
├── data/ # Reference data (geo targets, languages)
└── requirements.txt # Python dependencies
Run the interactive demo to see MerchVibeX in action:
python demo.pyThe demo showcases:
- System architecture and data flow
- Key features and capabilities
- Setup workflow steps
- Use cases and applications
- Technology stack overview
Validate your dataset health:
python data_quality_checker.py \
--project_id=<YOUR_PROJECT_ID> \
--dataset_id=merch_vibe_xMonitor data transfer status:
python monitoring.py \
--project_id=<YOUR_PROJECT_ID> \
--dataset_location=usThese tools help ensure:
- ✅ All required tables exist and are populated
- ✅ Data is fresh and up-to-date
- ✅ Data transfers are running successfully
- ✅ System health is optimal
The script uses Google Cloud Application Default Credentials. Make sure you're authenticated:
gcloud auth application-default loginBy default, the dataset is named merch_vibe_x. You can customize it:
sh setup.sh --project_id=<PROJECT_ID> \
--dataset_id=<CUSTOM_DATASET> \
--merchant_id=<MERCHANT_ID> \
--ads_customer_id=<ADS_ID>- API Documentation: See inline code comments for detailed function documentation
- SQL Queries: Review
sql/directory for query logic - Best Practices: Check individual module docstrings
Contributions are welcome! Feel free to:
- 🐛 Report bugs
- 💡 Suggest new features
- 🔧 Submit pull requests
- 📖 Improve documentation
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
theyashdhiman04
- GitHub: @theyashdhiman04
Built with ❤️ using Google Cloud Platform services. Special thanks to the open-source community for inspiration and tools.
This project is provided as-is for educational and commercial use. Ensure you comply with Google's API Terms of Service and your organization's data policies.
Made with ⚡ by theyashdhiman04
⭐ Star this repo if you find it helpful!