Jordan's Adventures Through Web Automation
Most of this process is documented really well by AWS here – https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html The most important parts that I believe are missed are when it
Demo code here Uploading directly to Lambda has a size limit of 50MB. This makes it difficult for Puppeteer, which is a very large library.
The struggle with Utah secretary of state continues. And it seriously is a struggle. I’m having real concerns about the successful completion of this scrape.
Puppeteer has deprecated waitFor! But it’s not as terrible as you think. There it is! Looking good!
Here we go! This is the first of a few parts of scraping the Utah Secretary of State. Utah uses a pretty complicated way to
Sample code here Another quick video showing the basics of web scraping with Puppeteer.
Sample code here Very basic code showing how to web scrape with Nodejs and Cheerio.
Sample code here This shows how to use Puppeteer with a proxy using both Luminati and Proxycrawl. Video guide: Luminati is a more intense solution
Very frustrating slight echo in this video. Sorry! Puppeteer on Lambda. Can it be done? Yes! I ended up using the chrome-aws-lambda package. It was
This is part 2 of the video to scrape North Carolina. This time finished off the scrape and getting the data. I also was able
