Crawler | Netpeak Spider
Short description : Crawler
Netpeak Spider is software with a powerful crawler that scans every page of a website to reveal issues that are preventing the website from ranking high in search.
Long Description : Crawler
Description Netpeak SpiderIf you have a website online, it would be interesting to browse it very often to see if everything is working correctly. This is precisely the role played by Netpeak Spider, a great tool that helps you crawl your website in just a short time. To understand how Netpeak Spider works, I invite you to read this description carefully.
What is NetpeakSpider ?It is an even more essential tool during certain web marketing activities such as:
- The creation and improvement of blog content;
- Creating or improving the link profile;
- The launch of an advertising campaign.
- List of incorrect redirects;
- Display of duplicate meta tags;
- Analysis of broken links;
- Collect all pages of a website and check them using 50+ SEO parameters;
- Detection of more than 50 on-page SEO issues and the priority level of each, as well as best practices;
- Analyze all incoming/outgoing internal links;
- Obtain the entire structure of the website;
- The determination of the internal PageRank, that is to say the capacity of each page in terms of authority;
- Professional management and validation of sitemaps;
- A detailed review of HTTP headers and source code;
- Use advanced settings to scrape any website;
- Filter, sort and segment data in the results table;
- Export over 70 reports, including the Optimization Quality Express Audit to PDF;
- Configure HTTP request headers.
How to configure Netpeak Spider ?To configure the Netpeak Spider explorer, you must go to the ”Parameter” tab of the tool. At this level you will find the many options that you can customize. The tool therefore offers quite specific parameters that allow you to carry out analyzes according to your needs. For example, if you care most about the content of your pages, just select this setting alone and turn off everything else. This makes the crawler’s site crawling process even faster and more efficient. If you are not an SEO newbie, you certainly know that it is important to create custom templates with specific settings to perform suitable tasks. When you set up the tool once, it only takes a few clicks to adjust it to perform other tasks as needed. In the settings, the tool offers a restriction option that lets you set custom issue restrictions. For example, if you are not satisfied with the response time of 2 seconds, you can set this as a problem in the settings.
How to Configure Netpeak Spider Crawl ?As in the previous section, the crawler settings are even more customizable. You need to be careful here not to get lost in the huge options you will have to configure. But, the advantage is that each option to configure is accompanied by an information bubble describing what it knows. The first two settings tabs called ”General” and ”Advanced” contain all the main exploration settings. Here are some important configurations you can do here:
- Crawl speed, it’s actually to set the number of concurrent threads, delay between requests, maximum response delay;
- Crawl Settings is the option to crawl exact website directory only, crawl all subdomains, crawl external links and check different file types;
- Consider crawling and indexing instructions like robots.txt, meta robots, X-Robots-Tag, etc;
- Crawl URLs from the <link> tag
- Other things like pausing, using cookies, and crawling relative canonical URLs;
- Enabling multi-domain exploration to explore different domains simultaneously.
Some Features of Netpeak SpiderHere are some data you can get when you run the tool:
DashboardThe dashboard is the first thing you can focus on when using these kinds of software. As for Netpeak Spider, here is what the dashboard has in store for us:
- Number of URLs crawled;
- Number of internal URLs;
- Number of URLs with errors and warnings;
- Number of conforming URLs.
- Eight blocks with a visual representation of key data;
- The percentage of compliant URLs,
- The severity of the problem;
- Status codes;
- The number of URLs per click depth;
Categorization of problemsTo allow users to have an easier understanding of the data, Netpeak Spider offers three different categories of problems according to their intensity:
Filtering, grouping and segmentation of resultsSince the tool provides a significant amount of information, it would be more interesting to have them in an order so as not to unconsciously skip the most important information. For this, the Netpeak Spider report gives you the possibility to sort the data, group the results according to certain parameters and create data segments. Here are some filter manipulations:
- Click on any issue in a sidebar to see only the pages that contain it;
- Click on any parameter of the “Overview” tab to see the pages that correspond to it;
- Click on any part of the website in the “Site Structure” report to see the pages of the corresponding part;
- Click on any value in the dashboard;
- Right-click on any value in the results table and select ‘Filter by value’;
- Define multiple custom filter rules.
Videos : Crawler
Images : Crawler
Company : Crawler
Netpeak Spider belongs to NetSpeak Group which is a group of companies domiciled in Ukraine. Specialized in the proposal of digital marketing services, SaaS software and B2C solutions, the company founded in 2006 has several missions, the main ones being:
- Ensure satisfactory profitability of companies;
- Support SMEs in the development of their activities;
- Constitute a model for the environment;
- Improve the state of mind of each other.