Fetch the homepage of this website the way the server delivers it, without
JavaScript, and give me the list of internal links it contains.
In September we put a new entry page live on our own site. Done properly: original text, in the sitemap, indexable, server rendered, fast. Eleven days later we asked Search Console how it was doing. The answer: the URL is unknown to Google.
Not "crawled but not indexed". Not "discovered". Unknown. Google had never seen the page.
What was not wrong
First the suspects it was not, because those cost you the least time to rule out.
The page was in the sitemap. The sitemap had been fetched. There was no noindex, no rule in robots.txt, no canonical pointing somewhere else. The page returned a 200 and served complete HTML from the server, so there was no blank screen for a crawler to choke on.
And one detail that put us on the trail: the English version of the same page was indexed. Same build, same sitemap, different result.
What was wrong
Our navigation is a dropdown menu. That menu is a component built in the browser: the links inside it only exist after JavaScript has run. They are not in the HTML the server delivers.
And that new page was only linked from that menu. Nowhere else on the site was there a link to it. So for anything reading the server HTML, no reference to that page existed at all. Only a line in the sitemap.
Google can execute JavaScript, that is true. But that happens in a second pass, and when that pass arrives is not up to you. A page that depends on that second pass for its only inbound link can wait a long time.
A sitemap says a URL exists. A link says it belongs somewhere. Those are not the same signal, and in this case the difference was eleven days and no indexing.
How to check this on your own site
This takes two minutes and needs no tooling. Fetch the page the way a crawler gets it and look at which links are in it:
curl -s https://www.yoursite.com/ | grep -o 'href="/[^"]*"' | sort -u
If an important page is not in that list, it is invisible to the first pass. Do it for your homepage and for the two or three pages where your traffic arrives.
The second check you do in Search Console, with URL inspection. That tells you not only whether a page is known, but also which page it was found through. If nothing is listed there, you no longer have a link problem but a link absence.
What we did about it
The fix is not to rebuild the menu. A dropdown menu built in the browser is a perfectly good menu, and it is there for visitors, not for crawlers.
The fix is to link the page from running text. We added four links to it: one in the introduction on the homepage, two in knowledge base articles on the same subject, and one in a blog post. All of those links are in the HTML the server delivers, because they are simply text.
After that we requested indexing by hand in Search Console. That cannot be done through an API, so it is one click, and above all something you do not need if the links are there from day one.
The rule we took from it
A new page gets at least one link from running text before it goes live. Not instead of the menu, but alongside it. That is not an SEO trick, that is how a site is supposed to work: if you cannot refer to a page in a sentence anywhere, the question is why that page exists.
The nice part is that the rule also tells you something about your content. A page that does not fit logically into any text is usually missing a clear subject.
Common mistakes
"It is in the sitemap, so Google will find it." A sitemap is a list, not a recommendation. It helps with discovery, but a URL without inbound links stays a URL nothing points at.
"Google renders JavaScript, so it does not matter." That it can does not mean it happens quickly. Everything you can hand over in the first pass, you hand over in the first pass.
"I check in my browser whether the links are there." In your browser JavaScript has already run, so you always see everything. Fetch the HTML without a browser, or switch JavaScript off.
"I will put a link in the footer on every page." That works technically, but a footer with thirty links means nothing to a visitor. One link in a sentence that is actually about something is worth more than five in a list.
"I request indexing and then I am done." Then that one page is helped. The next page you only put in the menu has the same problem. Fix the way you work, not the page.