Best of Crawling2021

  1. 1
    Article
    Avatar of phProduct Hunt·5y

    Automa - A chrome extension for automating your browser

    An extension for automating your browser by connecting blocks. From auto-fill forms, to scraping data of the website, it's up to you what you want to do with this extension. It's free to download and use, and there are no plans to stop using it.

  2. 2
    Article
    Avatar of devdojoDevDojo·5y

    How to Build a Web Scraper With Node.JS

    This tutorial will demonstrate how to automate scraping data off of the website and using it for whatever purpose. The project will be built with Node.js and Express. The source code of the completed project can be found on CodeSandbox. Fork, tweak the scripts and run the code.

  3. 3
    Article
    Avatar of smashingSmashing Magazine·5y

    How To Build An Amazon Product Scraper With Node.js — Smashing Magazine

    Web scraping is the practice of extracting large amounts of web data through the use of software. Scraping Amazon product pages can net you invaluable data such as features, prices and reviews. While the act of scraping publicly available data is legal, Amazon has some measures to prevent it on their pages.

  4. 4
    Article
    Avatar of dzDZone·4y

    Is Web Scraping Legal?

    The stance of the internet law on the legality of web scraping remains yet controversial. Most of the past legal face-offs between companies on web scraping ended up leaving traces of mind puzzles. With the court twists involved, a claimant might even end up guilty despite suing others for scraping their website.

  5. 5
    Article
    Avatar of freecodecampfreeCodeCamp·5y

    How to Scrape Websites with Node.js and Cheerio

    .com. How to Scrape Websites with Node.js and Cheerio. Joseph Mawa explains how to scrape websites with the two tools. He says it's easier than it looks and it's faster than you think. He also shows you how to use Cheerios to scrape sites with the same code.

  6. 6
    Article
    Avatar of dzDZone·5y

    Developing a Web Crawler with Node.js

    For business owners, all the data publicly available on the internet can be beneficial. This data can be helpful to generate leads for service providers and reach a bigger audience. Along with that, the data can also be used for the deep learning project to train the model. To run this code, you need to install a node.js package called a crawler.

  7. 7
    Article
    Avatar of jsPlainEnglishJavaScript in Plain English·5y

    Web Scraping 101: Introduction to Puppeteer

    Puppeteer is a Node library that provides users an easy and accessible API to control Chrome or Chromium without an actual browser window (headless) We will write some code to perform login, data scraping, and pagination with Puppeteer on http://quotes.toscrape.com.

  8. 8
    Article
    Avatar of logrocketLogRocket·4y

    Building a web scraper in Go with Colly

    In this tutorial, we will be taking a look at a Go package that allows us to build web scrapers, Colly. We will be building a basic web scraper that gets product information from an ecommerce store and saves the data to a JSON file. To follow along with the tutorial, you need to have Go installed on your local machine and you need at least a basic knowledge of Go.

  9. 9
    Article
    Avatar of jsPlainEnglishJavaScript in Plain English·5y

    How to Bypass CAPTCHAs with JavaScript

    How to Bypass CAPTCHAs with JavaScript, Puppeteer and 2Captcha. We will be automating the process of creating a new Reddit account, and in doing so we will be bypassing the common ‘I’m not a robot’ CAPTCHA.

  10. 10
    Article
    Avatar of inPlainEngHQPython in Plain English·4y

    Web Scraping with Selenium in Python

    Learn to identify elements to get content, take screenshots and run JavaScript. Change the size of the window and capture screenshots. To prevent blockages, use proxies and special headers. With our headless browser, you can do all of that and more. Python3 is required for the code to function.

  11. 11
    Article
    Avatar of jsPlainEnglishJavaScript in Plain English·5y

    The Easiest Way to Build a Web Scraper Using JavaScript and Node.js

    The Easiest Way to Build a Web Scraper Using JavaScript and Node.js Dan Suciu shares his tips on building a web scraper using NodeJS. In the following section of this article, I will help you write your own web scraper application using Nodejs.

  12. 12
    Article
    Avatar of jsPlainEnglishJavaScript in Plain English·5y

    PDF Generation in Node.js

    Puppeteer is a Node library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol. Using a puppeteer you can render the document and then generate PDF File from it. PDF make is a lightweight pure javascript library used to generate pdf.