How to Build a Sitemap

As indicated by the name, sitemaps serve as a map of a website. These maps help site crawlers from Google and other search engines navigate a website and accurately report information. By making it easier for crawlers to look through your site, sitemaps can improve your site’s SEO rating. Sitemaps provide multiple indexes that are sorted by category and within these indexes are URLs related to the index. For instance, if an index is site posts, then the URL of every post on a site will be sorted into that index. Alongside these URLs is metadata, which provides information such as the date the post was published and the URL’s relative importance compared to the rest of the site. This organization and information inform crawlers of sites available for crawling and provide a quicker means of crawling.

Do I really need a sitemap?

No one really “needs” a sitemap, but a sitemap can be an invaluable tool for improving your site’s SEO. Ideally, all large websites should have a sitemap, large being any website with over 500 pages. However, sitemaps can be beneficial to all sites regardless of size, and there is no risk because Google does not penalize for having a sitemap. If your site is new and has no external links (links from other sites to yours), it may be of greater importance to create a sitemap. This is because Google Crawlers work by following links from one website to another to scan and discover websites, and if no sites link to yours, crawlers may never find it. A sitemap increases the probability of a web crawler discovering your website.

It is also important to note that you may not need to make a sitemap yourself. Some web hosts create sitemaps for their clients. Simple web host services like Blogger and Wix may offer this function, so you will want to check. According to Google, you can “search your service’s documentation for the word ‘sitemap’ to see if a sitemap is generated automatically.” If you find that your service does build a sitemap for you, you are good. If your service does not build a sitemap for you, see if you can find out how your service would like you to submit a sitemap to them.

How Do I create a sitemap?

Before you even begin to create a sitemap, you need to decide which pages Google should crawl. The pages that you choose should be those that are most important to your site. Such pages would include your homepage and defining pages of your site, as well as, pages containing high-quality content (content that is most likely to drive traffic). You will also want to identify content to exclude from your sitemap, such as pages containing canonical versions of content (duplicate content). You will also want to look out for “thin content”: low-quality content: content that will not rank well and is least likely to drive traffic to your site. Do not include password-protected content that crawlers can not access either.

Once you have identified the content you want to include and the content you want to exclude, you can begin the sitemap building process. Unless you own a blog or google site, the odds are that you want to create an XML sitemap, which includes the location of a single site URL, as well as relevant metadata, such as the last time the page the URL points to was modified. In all sitemaps, you will want the metadata to include the last time a page was modified. This is important in case you want Google to re-index some of your site pages. You will also need to be sure that your sitemap contains no more than 50,000 URLs. However, it may be beneficial to create multiple sitemaps, each containing 1,000 or fewer URLs, unless your site is massive. This makes it easier and quicker for crawlers to scan your website. It may be easiest to separate these sitemaps by category. For instance, one sitemap could include all the important posts on your site, and another could include all the pages on your site. These multiple sitemaps could then be uploaded to a sitemap index file, which points to all the sitemaps on your site.

Now that we have the basics and general guidelines down for creating a sitemap, it is time to discuss creating a sitemap itself. There are two ways you can create a sitemap, one is manually, and the other is automatically through some online service. Both have their benefits and drawbacks. Creating a sitemap manually is time-consuming, but you will know the sitemap is done exactly how you want it. Creating a sitemap through an automatic generator is much quicker, however, the sitemap will not be made in the exact way you want it nor in the most optimal manner.

Building a sitemap manually

Choosing to build a website is certainly more of a challenge, but it is certainly not impossible. You may even be forced to build a sitemap from scratch if your site does not run on a CMS that supports a sitemap generator.

As previously mentioned, you can not have more than 50,000 URLs per sitemap, and you can not have more than 50,000 sitemaps in a sitemap index. This will be important to remember as you build your sitemap. You may also want to decide upon a categorization scheme before you start creating your sitemap, so you do not end up with duplicate URLs. Only the manual building of an XML sitemap will be discussed because XML is the format used to submit to search engines.

This manual process of building a sitemap will require some XML coding, but it is nothing too complicated and we will cover everything you need to know. Before getting started, it is important to note the necessary sitemap tags.

Opening tag Closing tag Description
<urlset> </urlset> Marks the beginning and end of your sitemap, all URLs within a particular sitemap will be located between these tags. Directly after the <urlset> opening tag, a namespace protocol will be necessary, which will be discussed later.
<url> </url> Specifies a url within a sitemap. Information about a url, including the url itself will go between these tags.
<loc> </loc> Written on the line below. The url being described by the <url> tags goes between these tags.

These are the most necessary and basic tags used within a sitemap. Next, we will discuss metadata tags that go between the <url> tags to describe a url.

Opening tag Closing tag Description
<lastmod> </lastmod> Includes the date of the last time a page the url leads to was modified. Should be written in the format YYYY-MM-DD
<changefreq> </changefreq> This provides a rough estimate of how often the page a url points to changes. Values for this tag include: AlwaysHourlyWeeklyMonthlyYearlyNever Where “never” indicates an archived page and “always indicates a page that is changed every time it is accessed. This value does not necessarily influence how often a crawler crawls a page
<priority> </priority> Can range from 0.0 to 1.0 and specifies how important a page on your site is. This value does not compare to other sites and priorities of pages are relative to each other on your site.

Now that you know the tags, you can start building your sitemap. Note that the metadata tags mentioned in the second table is not necessary, but that all metadata tags and the <loc> tag must go between the <url> tags. Here is an example:

It is also important to note that all URL addresses must be UTF-8 encoded. Provided here is a list of all characters and their encoded form in a site address (make sure you are looking at the NON-ASCII list). The only characters that should be encoded are those listed after your main site name that are non-ASCII, which in AOJ’s case, https://objectivejournalists.com/ would not be encoded, but all that follows would. Most URLs, however, will not need to be put through any encoding and encoding can be done when the xml file is saved.

To create your XML file and begin sitemap creation, you will first need to make a text file in a folder on your computer. Start by opening up the folder you want your sitemap in. Then right click in the blank space within your folder. A menu will appear, and from that menu you will want to click new and from the menu that pops up select text document.

Name this text file whatever you want, it will be your sitemap, then open it up.

Once the file open, you can begin typing in the code for your sitemap using the tags and example provided. Note that at this point the file is a text file, not an xml file, and you will not want to save it as an xml file until you are sure you are finished. Also, at the start of your sitemap file you will want to include

<?xml version=”1.0″ encoding=”UTF-8″?>

as shown in the example. Then you will want to type <urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″>

These two pieces are crucial to your sitemap, so be sure they are included. Continue building your sitemap by incorporating URLs of site pages until you are satisfied, remember, no more than 50,000 URLs per sitemap. When you finish, be sure to double check because once you save the file as xml there is no fixing. In fact, I would recommend you duplicate the file and keep a text version in case anything is wrong. When you are sure you have finished your sitemap, click File => save as and instead of.txt at the end of the file, put .xml. Also, be sure the encoding at the bottom is set to UTF-8.

Now that you have saved your XML sitemap, you must upload it to your site so that it may be used when submitting it to a search engine. This will vary depending on your website host. I would suggest you do a quick Google search to find out how for your website. You could even see if there is a YouTube video to show you how. However, if you use a WordPress site, upload your sitemap to your site by doing the following:

Log into the cPanel of your site by adding /cpanel to the end of your site address, for example:

www.example.com/cpanel

Then open up file manager in the files section. Once in file manager, open the folder that says public_html and click upload at the top. Then upload your XML sitemap file and remember the name of that file. When the file has finished uploading, open up a new tab in your browser and type your sites name with a /filename that file name is the one I told you to remember. For instance, if there was a site called www.coolcats.com with sitemap called sitemap.xml, the address to type would be www.coolcats.com/sitemap.xml. If there are any errors, something went wrong in the coding of your xml file when you typed it, so you will need to go back and fix such errors. However, if your sitemap uploaded correctly, it should look something like this:

Do not worry about the top warning stating that there is no style information associated with the XML file. Optionally, XML files can be styled, but search engines do not care about this.

Now that you have uploaded your sitemap, the only thing left to do is submit your XML sitemap to the web, which will be discussed at the end of this article.

Building a sitemap with an automatic generator

Building a sitemap with an automatic generator is far easier than manually creating one, however, the sitemap will not be done exactly how you want it and you will need to look out for a few things. First of all, unless you own a blog, you will likely want to use an XML site generator. Secondly, this generator must create an XML file that is UTF-8 encoded, otherwise, it will not be accepted by search engines. Thirdly, all generated xml sitemaps must include the URL of each page, however, you may want them to include more information like the date of last modification and priority of each URL. It is important to note that the priority of pages is only relative to other pages within your site.

Now that you know of a few things to look for, you can find an automatic generator for your sitemap. To avoid viruses, you will likely want an online generator that does not require download, or if it does, it is downloaded from a trustworthy source. If you own a WordPress site, one great plugin that generates a sitemap for free is Yoast SEO.

First download YOAST SEO from the plugins page on your WordPress site. Then, once you activate the plugin and have it all set up, scroll down on the backend sidebar and click SEO, as highlighted in blue below. Then, on the page that opens up, make sure XML sitemaps is on.

Once you are sure the Yoast SEO sitemap is on, type your site’s address into the search bar and at the end put “sitemap_index.xml”. Another option is to click the circle with the question mark next to XML sitemaps and to click the link that says, “see the XML sitemap”.

There are plenty of other sitemap generators on the web as well for those without WordPress. However, be sure you look carefully and check that the sitemap is safe and includes what you want.

Submitting Your Sitemap

There are two ways you can submit a sitemap to Google. The first way is a little riskier, especially if you are not the most technologically savvy, and it is to add your sitemap to your robots.txt file. The second way is less risky and allows you to view a bunch of other useful information about your site, and it is to submit your sitemap to Google Search Console.

Starting with the robots.txt file method. Log in and access your website’s folders via a file manager. If you have WordPress, you can do this by logging into cPanel by adding /cpanel to the end of your site URL and then opening up file manager. Once in file manager, open up the folder that says public_html and click on robots.txt within that folder. Now click to edit this file. In WordPress, you will be prompted with a warning, but must click to proceed anyways. Anywhere with this text file, so long as it does not interrupt existing text, type

Sitemap: URL of your sitemap

The URL of your sitemap is the address you type in to view the sitemap of your website. An example of what you would type is:

Sitemap: http://example.com/sitemap_location.xml

Once you do this, you have successfully submitted your sitemap to at least the Google search engine.

Now for the Google Search Console method. If you do not already have a Google Search Console account, you will first need to set one up, which may require that you wait one to two days before submitting your sitemap. First, head to https://search.google.com/search-console/about and click “Start now.” Then, enter your URL with the https:// or http:// into the option that says URL prefix and click continue. A box will show up with multiple options to verify your site, choose whichever seems easiest for you. If one method does not work, try another.

If you have a WordPress site and want to use the HTML file, you can do so by downloading the file given and logging onto your site’s cPanel by typing /cpanel at the end of your site address. For example:

www.example.com/cpanel

Then click on file manager, it should be located in the files section. In file manager, open the public_html folder and click upload at the top. Drag the html file you downloaded from Google Search Console into the upload zone and when the download gets to 100%, go back to Google Search Console and click verify. You should be in now.

If you are using the DNS method and own a WordPress site, copy the text provided from Google Search Console. Then, just like for adding the HTML file, log into cpanel. This time, we will not open file manager, but instead will open zone editor under the domains tab.

Then, for the correct domain, click manage.

Once in the records of the domain, click the blue add record button. For the Zone name, write the name of your website without the www or https://. Keep TTL at 14400, but change type to TXT and paste the text provided from Google Search Console into the record field. When you have finished filling out all fields, click add record.

Now you can go back to Google Search Console and verify. If the verification does not work, wait one day and try again.

Once you have successfully signed up for Google Search Console and been verified, log into your Google Search Console and click on sitemap on the sidebar. Then enter in the name of the sitemap file you uploaded to your website if you did so manually. Otherwise, type in everything past your sitename that was included in the URL address to reach your website’s sitemap. For instance, if the URL to a sitemap was www.example.com/sitemap.xml, the only thing to type would be sitemap.xml. Once you have entered your sitemap URL, click submit. You have now successfully uploaded your sitemap to Google. For other search engines, you will need to read up on how to submit your sitemap, but it should be a lot easier now that your sitemap has actually been created.

Feel free to leave comments for any additional questions you may have.

Citations:

Protocol. (n.d.). Sitemaps.org – Home. Retrieved February 26, 2020, from https://www.sitemaps.org/protocol.html

Knowledgebase. (n.d.). Client Area – EverWeb. https://billing.ragesw.com/index.php/knowledgebase/113/This-XML-file-does-not-appear-to-have-any-style-information-associated-with-it.html

Google. (n.d.). Build and Submit a Sitemap. Google Help. Retrieved February 26, 2020, from https://support.google.com/webmasters/answer/183668?hl=en

Leave a Reply

Your email address will not be published. Required fields are marked *