Get Company Official website URL using Company Name in Python

In this blog, we will explore how to retrieve a Company’s Official Website URL in Python. Whether you’re building a web scraper or automating search queries, knowing how to get a company’s website from just its name can be handy. We’ll cover methods using APIs like SerpApi and web scraping tools such as BeautifulSoup and requests.

What You Will Learn:

Using SerpApi to get the website URL via Google search.
Scraping search results from Bing using BeautifulSoup.

Key Requirements

Before diving into the code, you need to install the required libraries. For this blog, we’ll be using:

SerpApi for searching Google in a structured way.
requests and BeautifulSoup for scraping search engines.

Let’s get started!

Method 1: Get Company Official Website URL in Python Using SerpApi

To retrieve a company’s official website using SerpApi, we’ll use Google’s search engine results in a structured way.

Step 1: Install SerpApi Python Client

First, install the google-search-results Python package:

pip install google-search-results

Step 2: Get the API Key

Step 3: Code to Get the Company Official Website URL in Python

Here’s a Python function to search for a company’s official website

from serpapi import GoogleSearch

def get_company_website(company_name):
    params = {
        "engine": "google",
        "q": company_name + " official website",
        "api_key": "YOUR_SERPAPI_KEY",
    }

    search = GoogleSearch(params)
    results = search.get_dict()
    
    # Extract the website URL from the first result
    for result in results.get("organic_results", []):
        if "link" in result:
            return result["link"]

    return "No website found"

company_name = "Microsoft"
website_url = get_company_website(company_name)
print(f"Official website of {company_name}: {website_url}")

n this code:

GoogleSearch() performs the search query.
get_dict() fetches the search results.
It extracts the first relevant link that points to the company’s official website.

Pro Tip: Replace "YOUR_SERPAPI_KEY" with your actual SerpApi key.

Method 2:Company Official Website URL Using Web Scraping with BeautifulSoup

If you don’t want to use an API or need a free option, you can scrape search engine results using requests and BeautifulSoup.

Step 1: Install Required Libraries

Install the requests and beautifulsoup4 libraries if you haven’t already:

pip install requests beautifulsoup4

Step 2: Python Code to Scrape Bing and Get the Website URL

Since scraping Google is against their terms of service, we will use Bing for this example.

Here’s how you can scrape the search results from Bing and extract the official website:

import requests
from bs4 import BeautifulSoup

def get_company_website(company_name):
    search_url = f"https://www.bing.com/search?q={company_name}+official+website"
    response = requests.get(search_url)
    soup = BeautifulSoup(response.text, 'html.parser')
    
    for link in soup.find_all('a', href=True):
        href = link['href']
        if 'http' in href and company_name.lower() in href.lower():
            return href

    return "No website found"

company_name = "Microsoft"
website_url = get_company_website(company_name)
print(f"Official website of {company_name}: {website_url}")

Step 3: Understanding the Code

requests.get(): Sends a request to Bing’s search page for the company name.
BeautifulSoup: Parses the HTML document and helps you extract all hyperlinks from the search results.
The script checks for the first link that contains the company name and extracts the URL.

Note: Make sure you follow the search engine’s terms of service when using web scraping methods.

Conclusion

Now you know how to fetch the official website of a company using Python! Whether you use an API like SerpApi for structured Google search data or prefer web scraping using requests and BeautifulSoup, these methods provide a practical way to automate search queries for company websites.

For more Python tips and projects, be sure to check out our Python Projects section and follow our Beginner’s Guide to Web Scraping.

FAQs

1. What is SerpApi?

SerpApi is a service that provides real-time structured search engine data from sources like Google, Bing, and others.

2. Can I scrape Google search results?

Scraping Google directly is against their terms of service. It’s better to use an API like SerpApi for accessing Google search results.

3. Which libraries are best for web scraping?

requests for HTTP requests and BeautifulSoup for parsing HTML are popular choices for web scraping in Python.