Man standing in front of a computer watching checking that his sites were submitted to search engines properly

How to Submit Your Website to Search Engines

In this article I’m going to cover a lot of things but the major takeaways will be:

  • How to submit your website to the various major search engines.
  • How to check and make sure your website (or new website pages) have been indexed.
 

If search engines (like Google and Bing) don’t know your website exists then your website will never appear in any searches.

That means you’re throwing away a lot of opportunities for people to find your site and visit it. 

If your website is a part of your business and you depend upon people visiting it to generate revenue, that means you’re throwing away a lot more than just opportunities: you’re throwing away money.

Your website doesn’t just automatically appear on Google and the other top search engines by magic either.

In fact, several weeks can pass by without the search engines ever knowing your entire website exists.

We both know that’s a problem. So what can you do about it?

Why Do I Need to Submit My Website to Search Engines?

Search engines like Google have programs called bots that crawl the web and create massive indices of the world wide web.

These bots primarily rely on links from one page to another to discover them and index them. They’re pretty damn good at this but as you can imagine, the world wide web is huge. 

If links to your site don’t exist on other pages already in that search engine’s index then you’re site won’t get automatically indexed (link building should be a part of your SEO campaigns).

What Does Crawling a Website Mean?

Imagine you’re on Wikipedia reading an article article about Abraham Lincoln. While reading the article you see a link to the Gettysburg Address and click on it because you want to read more about it.

That same concept is how bots crawl the web. They visit web pages and follow the links on those pages.

As a page gets crawled by a bot it gets indexed in the search engine. 

Now, there’s a lot of nuance that we haven’t covered but if you want to learn more about it then you can watch this video by Matt Cutts, the former head of Google’s webspam team.

Reasons Why You Should Still Submit Your Website to Search Engines Manually

 
  1. Low effort, high reward. There’s no excuse not to. It’s just too easy to submit your website to Google and the other major ones for indexing.
  2. Search engines aren’t perfect. They can’t find everything on their own. Submitting your website manually makes it easier for the major players to begin ranking your website for keywords. By submitting your website manually you can also give the search engines information they couldn’t figure out on their own such as how important each of your website’s pages are.
  3. You’ll get access to some pretty helpful dashboards of data. Both Google and Bing each have dashboards you can use to gain insight into how you’re performing in the world of search. You’ll get access to some pretty useful tools also.
 

Time for the bad news. Don’t be fooled into thinking that you’ll automatically start ranking for keywords after getting your site indexed. 

In fact, most sites don’t rank on the first page of Google for major search terms for a very long time.

So if you thought this was a magic bullet. Sorry. It’s just the first step.

How to Submit Your Website to Google

Performing a search engine submission with Google is pretty simple. Just follow the instructions below.

To submit a website URL to Google you need to use a website/tool called Google Search Console (formerly Google webmaster tools). You’ll need a Google Gmail account to use Search Console (and basically all Google Tools) so either sign in or create one.

Once signed into Search Console you have two options to submit your site for indexing.

  1. Submitting an XML Sitemap
  2. Typing the URL into the search bar at the top of the page.

What's an XML Sitemap?

An XML sitemap is a file that lists all of the page URLs of a website in XML (eXtensible Markup Language) format. XML is just a file format, like PDF or JPEG. 

If the sitemap has been formatted for easy readability it will look something like this:

XML Sitemap

How Can I Check if my Website has a Sitemap?

Check yourdomain.com/sitemap.xml

Your sitemap xml file is usually located at yourdomain.com/sitemap.xml.

Example: Let’s say your website address is https://www.rundoyen.com/. Then you can check if your sitemap exists by visiting: https://www.rundoyen.com/sitemap.xml

If the page isn’t found that doesn’t mean you don’t have a sitemap. It’s possible that it has a different URL.

Check Your robots.txt File

You can also check if your sitemap has been listed in your robots.txt file. To check that try typing in yourdomain.com/robots.txt and you should see something like the following:

XML Directive in Robots.txt File

As you can see, this site has two sitemaps listed and neither are using the traditional /sitemap.xml path.

Use Advanced Search Operators

If your sitemap isn’t traditionally named AND isn’t in your robots.txt file, then you can also check Google’s search engine directly by using advanced search operators on your domain name.

Go to Google and use the following advanced search operator sequence to see if Google has your sitemap indexed:

site:yourdomain.com inurl:sitemap

Google Advanced Search Operators finding XML Sitemap

If you have located your sitemap then you can go ahead and submit it to Search Console.

Just go to Search Console → Select Your Website → Sitemaps → Paste Sitemap URL → Click Submit

Submitting Sitemap to Google Search Console

How to Submit Multiple Sitemaps

If you have multiple sitemaps you can just repeat this process for each sitemap URL.

It’s as easy as that.

If you already have your website’s sitemap submitted, then it will crawl your site at scheduled intervals.

What if you just published a new page or blog post you’re really excited about though? How can you get that submitted to Google faster? With a manual submission.

The submission process is easy, don’t worry.

Just visit Search Console and enter your URL in the search bar at the very top. If the page hasn’t been indexed yet Google will let you know and you can click the “Request Indexing” button to index the page to do so.

URL is not indexed in Google

If your page is indexed already, you’ll see results like this:

URL is Indexed by Google Search Console

How to Submit Your Website to Bing

You’ll need to use Bing’s Webmaster Tools (similar to Google Search Console) to submit your full sitemap to Bing for indexing.

URL Indexed by Bing

If you haven’t set up your Bing Webmaster Tools account yet you can authenticate it using Google. Let us know if you have any issues.

How to Submit Your Website to Yahoo

This one is the easiest. There’s nothing to do! Bing powers Yahoo so by indexing your site with Bing you’ll be indexed with Yahoo automatically!

How to Know if Your Website has been Indexed

You can use the advanced search operators we talked about earlier to check if your page has been indexed by Google.

Follow this syntax: site:yourdomain.com/page-subfolder-to-check/

Example: Let’s say we just published a new page on rundoyen.com that’s located at https://www.rundoyen.com/coach-match/ and need to check it.

We would type in the following:

site:rundoyen.com/coach-match/

If you do that and don’t get any results for that advanced search then your page likely has not been indexed by the search engine yet (the advanced search operator site: works for both Google and Bing).

It can sometimes take days for Google to get your page indexed so check back in a few days if it doesn’t appear when with the site: search operator after you’ve submitted the page to be indexed.

I’ve had pages get indexed within 30 seconds before (but not always). So it doesn’t hurt to check right afterwards.

Using Google Search Console to Check if your Site has been Indexed

Go to Google’s Search Console → Index → Coverage → Click on the Valid tab and then click submitted and indexed and Google Search Console will return a list of pages that have been indexed.

Checking index coverage in Google Search Console

What if my Website isn't Indexed?

There are a few common culprits that we can check if you have determined your website/web page isn’t indexed:

  1. Flagged with a noindex tag
  2. Blocked by robots.txt
  3. Blocked by .htaccess

Page Flagged noindex

The noindex meta tag directs the search engine crawlers not to index a certain page. This tag exists at the page level. Search through your page’s HTML to figure out if it has been flagged for no index.

The easiest way to do this is to open up the DevTools for the page and look for the term “noindex” using the search function (Windows: Control + F, MacOS: ⌘ + F ). You can learn how to use the DevTools in Chrome here.

Blocked by robots.txt File

If your site’s robots.txt file is blocking Google’s bot (Googlebot) from crawling your website, then you will want to fix that.

You can either review your robots.txt file directly by visiting yourdomain.com/robots.txt or using Google’s Robots Testing Tool.

Note that this tool is a part of the old Google Search Console and isn’t present in the current version of Google Console. You can still access the older Robots Testing Tool here.

Blocked by .htaccess File

If your site is running on an Apache-based server then you’ll have an .htaccess file in your root directory. Locate this file and look to see if the following is present:

Header set X-Robots-Tag "noindex, nofollow"

If you find that, then you’ll need to remove it to grant Googlebot access to crawl and index your website.

If this is above your technical skills just reach out to whoever maintains your website or feel free to ask us for help with technical SEO as well.

My Website is Indexed but Not Ranking

There are a lot of sites indexed in the search engines. Just go to Google and type in any term and you’ll see how many pages are indexed for that term.

Google Index Amount

Wow, “indexing with Google” has over 46 million results indexed.

But only 10 show up on the first page of Google.

When’s the last time you searched for something and checked the second page?

Exactly.

Pages that aren’t ranking on the first page of Google receive very little traffic. You really need to be in the top 3 results to capture significant organic traffic for a search term.

Getting to the first page of Google is one battle. Climbing the ranks from position #10 to the top 3 positions is another battle.

Getting to the first page of Google is, in my experience, primarily correlated with how many backlinks your website is receiving from other sites.

Backlinks will still play a role in climbing up the ranks from 10 to the top 3 but it is at this point that on-page SEO factors really begin to matter.

Be sure you’re tackling both objectives: backlink building and on-page SEO.

Final Thoughts

If you want to take things casually and aren’t all that concerned with ensuring that Google indexes your site you can simply do nothing. Odds are that Google will find your site eventually…

If you’ve spent money on creating a website, hosting it, etc. then it really doesn’t make any sense to not manually index it with the search engines. You’re not taking your ROI seriously if you don’t.

This is especially true if you run a business and depend upon your website for lead generation or sales.

As I said before though, indexing is just the first step. In most cases you’ll need to invest in SEO to get the results you want.

Skyler Reeves Transparent Background
Article by

A philomath who loves to teach. Often too sarcastic in meetings.  Head of Content Marketing & Search Optimization at Ardent Growth. CMO of RunDoyen. B.S. in Computer Science & B.S. in Philosophy from MSU. 

Share on facebook
Share on twitter
Share on linkedin

Tell us a bit about your Business

Fill in your details and we’ll get back to you in no time.