Web Scraping Blog - Cobalt Intelligence

Guides on how to obtain data from all over the web

Puppeteer

Secretary of State API

Get business data from the Secretary of States, all via API.
If you have a business name or entity id, you can automate this collection of business data with the Secretary of State API. These tutorials can help you make it happen.

        let url = `https://apigateway.cobaltintelligence.com/search?searchQuery=${encodeURIComponent(businessName)}&state=${state}`;

        if (liveData) {
            url += '&liveData=true';
        }

        const axiosResponse = await axios.get(url, {
            headers: {
                'x-api-key': this.apiKey
            }
        });

        // This will take longer
        if (axiosResponse.data?.retryId) {
            // Handle retryId
        }

        return axiosResponse.data;

Web Scraping Tools

Technical posts and videos addressing how to handle some of web scraping’s toughest challenges.

Real Estate

Posts and videos talking about how to get real estate data from the web.

Technical Secretary of State

In depth technical videos about how to scrape business data from various Secretary of State websites.