Creating the Perfect Sitemap

Creating the perfect sitemap can make a world of difference when it comes to the success of any website. At the same time, however, creating a sitemap can either be very easy or quite complex (and time consuming) depending on the website. A larger, more complex site with hundreds or thousands of pages, for instance, will be much more time consuming when it comes to creating a sitemap than a smaller one.

It can take hours. Hours that most people don’t have because there are more important things to focus on such as creating rock solid content, promoting your business or website, and so on. But at the same time, part of your success is having a website that is fully optimized for search engines and stellar when it comes to the user-experience.

Right? Well, the good thing is that a sitemap generator can cut the time down it takes to create the perfect sitemap considerably. In most cases, a sitemap generator is completely automated as well.  

Let’s get down to the nitty gritty of this thing called a sitemap, first. What is the purpose of a sitemap? And the big question: will it do my website any good? The short answer is a massive yes! In fact, many webmasters and SEO professionals have reported improved traffic after submitting a sitemap. In addition to the perks of utilizing a sitemap for search engine optimization, a sitemap can make a world of difference when it comes to the usability and accessibility of your website, ensuring that it’s usable for all visitors across the globe.

There are two types of sitemaps—HTML and XML—and quite frankly, in order to have the perfect sitemap (that does its job flawlessly), it’s recommended that your website has both. An HTML sitemap is intended for human visitors whereas an XML sitemap is made for search engine robots, and having both will help clarify the website’s purpose and goal, avoiding duplicate data and helping to minimize the number of steps it takes to travel from one page to another.

Before we get into creating the perfect sitemap, let’s talk about both HTML and XML sitemaps. A sitemap is basically a list of URL’s that describe a website’s section and pages to ensure that both users and search engines find specific content irrespective of their location on the website. Naturally, websites are crawled periodically by bots but this is unpredictable. Not only that, there is no guarantee that even during the crawling that all pages will be identified and indexed by search engines.

HTML Sitemaps

HTML Sitemap sare created and available for viewing by website visitors to help them navigate through a website. Usually, they are setup with a linear structure, showing the hierarchy of the site from top level pages to lower level ones. They provide the user with a very easy to read outline of the content as well, which makes navigation to their desired piece of content that much easier.

XML Sitemaps

XML Sitemaps, on the other hand, are a little different. These particular sitemaps are intended for search engines and spiders, not visitors. They can be visible with any web browser but their main function is to provide the URL’s of a website to search engines. They even show data on how often a page has been changed, compared to other URL’s on the same site. This information is very important for search engines because the more links they are provided with, the more a website will appear in search engine results.

There are quite a few ways to create a sitemap—both manually and utilizing a sitemap generator. The other great thing about using a sitemap generator is the possibility of creating an absolutely perfect sitemap. Because why? That is what these tools were made for, to create visual HTML and XML sitemaps that will get the job done.

Sitemap Creator Options

The following options below will help you to create the perfect sitemap with the sitemap generator, DYNO Mapper:

Creating from a URL:

Enter the base URL that you would like to create a visual sitemap for including the http:// or https:// located at the beginning of the URL.

Creating from XML:

Create visual sitemaps from xml files that are located on your computer or online via a URL.

Creating from Scratch:

Create visual sitemaps using the DYNO Mapper editor to manually add, remove, edit, and set categories of pages without the need to crawl URLs or import xmls.

Creating from Existing:

Create visual sitemaps from previously created sitemaps that are save in DYNO Mapper.

In addition to the above, you can create a visual sitemap from sitemaps that have been previously created within your DYNO Mapper account.

Advanced Options for Creating a Sitemap

DYNO Mapper offers some great advanced options for its users, even those who have taken advantage of their 14 day free trial, including:

• Follow robots.txt

• Follow subdomains

• Ignoring trailing slash

• Ignore query string

• Unique page title

• Ignore scheme

• Arrange links by URL

• Domain as root

• Website authentication

• Basic authentication

What does all of this mean?

Follow robots.txt

The robots.txt file allows websites to indicate to web robots like Google robot and DYNO Mapper that you would like them to ignore certain pages. The preference in DYNO Mapper is on by default and can be switched off in the Create from URL advanced options quite easily (see below). The robots.txt file name must be written in lowercase and placed in the root directory of your website and follow the Robots Exclusion Protocol.

There are two important considerations when using /robots.txt:

1. robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.

2. the /robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use.

Follow sub-domains

Now, a sub-domain is a domain that is part of a larger domain. For example, north.example.com and south.example.com are subdomains of the example.com domain. It’s essentially an easy way to create a memorable web address for unique content areas of your website.

Ignoring trailing slash

If website code is hand written, it may not be consistent and also contain certain problems even though the page will still appear to work properly in a web browser. The ignoring trailing slash option will allow the DYNO Mapper to be more forgiving and overlook problems like a web browser does if on is selected or be less forgiving if no is selected.

Ignore query string

A query string is info within a URL that follows a question mark (?) such as the one shown below:

www.example.com/index.html?thisis=thequerystring

The option is important if you have dynamic sections within your website, like a calendar. Dynamic sections within a website that use query strings can consist of many pages that you might want to omit from your sitemap.

Unique page title

If your website contains pages that each have a unique title, selecting “yes” for the unique page title option will give you a very accurate crawl. Some content management systems can refer to the same page by quite a few different URLs which causes duplicates within your visual sitemap.

Ignore scheme

With the ignore scheme option; some websites have duplicate pages represented because they allow multiple schemes of an individual page. For example, if you do not choose “ignore scheme” for www.example.com, http://www.example.com and https://www.example.com would be considered two different pages.

Arrange links by URL

If you would like DYNO Mapper to arrange your sitemap based on the parent/child relationship used in the URL, keep the “arrange by URL” option on.

Domain as root

If you are crawling a sub-domain, the “domain as root” setting lets you set that sub-domain as the root of the sitemap tree.

Website authentication

Use website authentication option if you would like to include private front end pages within your sitemap from popular CMS systems. If your favorite CMS system is not in the list, please send us an email to inquire if we can add it CMS systems that we support.

Basic authentication

Use the HTTP basic authentication for authentication that is static, standard HTTP headers.

DYNO Mapper makes clicking these selections on and off very easy after you have created the perfect sitemap. With that said, a sitemap generator is more than just a tool, it’s a powerful application that can change the way visitors view your site; making a world of difference in your conversions and profits, as well as your overall performance.

A sitemap generator will crawl your website and create an automated sitemap of up to 50,000 URLs and up to 10 megabytes. It will catch any of the pages you might have forgotten about, pull up any old event pages and allow you to simply add the pages it comes up with into a spreadsheet.