Introducing our Bing Webmaster Tools API Python Client

Managing Director

November 11, 2024 · 4 min read

For organisations managing large-scale SEO operations, accessing and analysing Bing Webmaster Tools ("BWT") data programmatically has become increasingly crucial. Today, we're excited to introduce our open source Bing Webmaster Tools Client written in Python for the Bing Webmaster Tools API – a modern, type-safe, and production-ready solution for accessing Bing's webmaster data at scale.

➡️ Ready to start? Head straight to Github...

Bing Webmaster Tools Github

Why Another Bing Webmaster Tools Wrapper?

With the rise of new ways to search, our focus has been on expanding data pipelines outside the Google Search ecosystem. Bing's significance has grown substantially, particularly with its integration with Copilot.

Google provides an official Python SDK for their APIs which includes Search Console (Github link). Bing has a Python SDK, but only for Bing Ads (Github link).

What began as an internal tool for our data pipeline needs has evolved into something we believe can benefit the broader technical SEO community. By open-sourcing our client, we aim to provide a robust solution that matches the quality and reliability of official SDKs.

Key Features

Async/await support - Built on aiohttp for efficient async operations
Type-safe - Full typing support with runtime validation using Pydantic
Domain-driven design - Operations organised into logical service groups
Comprehensive - Complete coverage of Bing Webmaster Tools API
Developer-friendly - Intuitive interface with detailed documentation
Reliable - Built-in retry logic, rate limiting, and error handling
Flexible - Support for both Pydantic models and dictionaries as input
Production-ready - Extensive logging, testing, and error handling

What Are The Use Cases For the BWT API?

The BWT API serves three main purposes: collecting marketing data for warehousing, monitoring search engine behaviour, and automating website maintenance tasks:

Bingbot Crawl Stats: This data is not accessible via the web interface. Obtaining server access logs for bot traffic can be time-consuming, so these stats provide some insight into potential issues.
Extracting Search Query Data: Up to 16 months of historical data can be backfilled, with daily data collection possible moving forward.
XML Sitemap Management: Some organisations have thousands of XML sitemaps. By running scripts alongside Google Search Console, both Google Search Console and Bing Webmaster Tools can stay synchronised. A single source of truth JSON file is typically built for this purpose.
User Management: Teams often handle hundreds or thousands of website domains. Compliance is maintained with a risk register (aligned with ISO 27001 and SOC 2 Type II) and a review process for adding and removing users.
Content Submission: While IndexNow is the preferred method for Bing and other search engines, it requires a static file on the site. Content Submission allows for pushing up to 10,000 new URLs per day to Bing, useful when development teams cannot prioritise the static file during a Sprint.
Blocking Content: Handles legal and compliance requests effectively by blocking web pages or previews.
Parameter Removal: Unlike Google, which no longer supports parameter management, Bing still allows it.
Regional Settings: Subdirectory-level regional settings can be configured when hreflang implementation poses difficulties. This option is not available via the web interface.

Technical Overview

Getting Started

To begin using the wrapper:

Install via pip:

1pip install bing-webmaster-tools

Add your BWT API Key to the environment variable using our authentication example.

1export BING_WEBMASTER_API_KEY=your_api_key_here

3. Look under the examples subdirectory to see boilerplate command line interface scripts which can help you get started. These assume you are using the environment variable BING_WEBMASTER_API_KEY as described in the Basic Setup.

1# Listing all your Bing website profiles
2python examples/get_all_sites.py
3
4# Do you have any blocked webpages that you don't know of?
5python examples/get_blocked_webpages.py -s https://merj.dev
6
7# Set the highest crawling available on a scale of 1-10 (default 5).
8# Check if "crawl boost" is available, which is valuable for very large websites.
9python examples/submit_crawl_settings.py -s https://merj.dev -r "10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10" --enable-boost
10
11# Get all your XML sitemaps, RSS Feeds, etc. 
12python examples/get_feeds.py -s https://merj.dev -v
13
14# Submit all your latest content
15python examples/submit_batch_urls.py --site-url https://merj.dev --input file.txt
16
17# Get all related words that contian a word.
18python examples/get_keyword.py -q "seo"
19
20# Get all 
21python examples/get_keyword_stats.py -q "seo"
22
23# Get all URLs that have been fetched through the web UI
24python examples/get_fetched_urls.py -s https://merj.com
25
26# Get a list of URLs with issues and a summary table
27python examples/get_crawl_issues.py -s https://merj.com
28
29# get 
30python examples/get_query_page_stats.py -s https://merj.dev -q "seo"
31
32# Get the stats for a page with a particular query
33python examples/get_query_page_detail_stats.py -s https://merj.dev -q "seo" -p "https://merj.dev/seo"
34
35# Get a summary of all external links to a webpage
36python examples/get_url_links.py -s https://merj.dev -l https://merj.dev/seo
37
38# Get 3 summary tables: Crawl Stats, Status Codes and Issues.
39python examples/get_crawl_stats.py -s https://merj.dev
40
41# Manage_parameters shows how you can use CLI args to perform different actions.
42# List all parameters
43python examples/manage_parameters.py -s https://merj.dev --list -v
44# Add a parameter
45python examples/manage_parameters.py -s https://merj.dev --add sort
46# Remove a parameter
47python examples/manage_parameters.py -s https://merj.dev --remove sort
48# Enable a parameter
49python examples/manage_parameters.py -s https://merj.dev --enable sort
50# Disable a parameter
51python examples/manage_parameters.py -s https://merj.dev --disable sort

Architecture and Approach

The client is built with modern Python practices using asynchronous functions. The following example creates the authentication from an environment variable, gets all the known sites, then loops through them to output the traffic stats.

1from bing_webmaster_client import Settings, BingWebmasterClient
2
3async def main():
4    # Initialize client with settings from environment
5    async with BingWebmasterClient(Settings.from_env()) as client:
6        # Get all sites
7        sites = await client.sites.get_sites()
8        if not sites:
9            print("No sites available")
10            return
11            
12        test_site = sites[0].url
13        print(f"Using site: {test_site}")
14        
15        # Get traffic stats
16        traffic_stats = await client.traffic.get_rank_and_traffic_stats(test_site)
17        print("Traffic Statistics:")
18        for stat in traffic_stats:
19            print(f"Date: {stat.date}")
20            print(f"Clicks: {stat.clicks}")
21            print(f"Impressions: {stat.impressions}")
22
23if __name__ == "__main__":
24    import asyncio
25    asyncio.run(main())

The client handles all the complexity of API authentication, rate limiting, and data pagination, allowing you to focus on utilising the data rather than managing API interactions.

Current Limitations and Roadmap

As our public v1.X.X release, it's important to note current limitations:

We are currently only supporting the BWT API key.
Our pursuit of creating the Python client also uncovered several BWT API bugs which are currently being addressed by the BWT team. Some BWT endpoints may not work as expected. We shall temporarily patch and work around these endpoints where possible.

Development roadmap for 2024/2025:

Expand mock testing for all endpoints
Add OAuth 2.0 integration
Add command line interface for endpoints
Improve documentation and examples
Remove deprecated endpoints in line with BWT deprecation
Work with the Bing Webmaster Tools team to improve API endpoints and documentation

Community and Contribution

We're excited to see how marketing technology teams and the SEO community build upon this library. Please feel free to submit:

Feature requests and bug reports via GitHub issues
Contributions through pull requests

Next Steps

Visit our GitHub repository to download the code and get started.
Join in the conversation on:
1. X using the hashtag #bingwebmastertools
2. Linked using #bingwebmastertools