Page 1 of 1
Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Tue Oct 04, 2022 5:35 pm
by IvoPetkov
Hi I am using opencart 2.1.0.2
I have a drop in organic search and some pages are disappearing on the SERP.
When analyzing the site with ahrefs, it gives me as the biggest problem that it cannot index the pages from the categories that are numbered 2, 3, 4, etc.
When the site was created 6 years ago in sitemap and robot.txt it was set to index only 1 page of the categories.
Should the sitemap and robot.txt be changed?
And if you have to, can you suggest any solutions that will correct the problem?
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Tue Oct 04, 2022 7:56 pm
by ADD Creative
If you want search engines to index the other pages you would need to change you robots.txt. It's usually a good idea to allow the other pages to be indexed so search engines will know how your products relate to your categories.
OpenCart sitemap generator will only list the first page. Search engines should find the other pages from the page links.
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Tue Oct 04, 2022 9:55 pm
by IvoPetkov
And how do I make robot.txt so that it both indexes the back pages and always shows the first page in SERP.
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Tue Oct 04, 2022 10:51 pm
by ADD Creative
Without knowing what's in your robots.txt it's impossible to say.
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Wed Oct 05, 2022 12:25 am
by IvoPetkov
This is only in the file
User-agent: *
Disallow:
Sitemap:
https://my-site.com/sitemap.xml
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Wed Oct 05, 2022 4:13 pm
by ADD Creative
That robots.txt isn't blocking anything.
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Fri Oct 07, 2022 3:59 pm
by IvoPetkov
It must have been done from the sitemap or directly in the hitaches file
Where should I look?
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Fri Oct 07, 2022 6:57 pm
by ADD Creative
If there is nothing in the robots.txt. You need to work out why the pages are not being indexed before you know what to look for. The Google Search Console could give you some details.
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Fri Oct 07, 2022 7:38 pm
by JNeuhoff
It would be useful to have a URL of your website, to see what's happening here.
Also make sure that your page header doesn't contain entries like these:
Code: Select all
<meta name="robots" content="noindex">
or
Code: Select all
<meta name="googlebot" content="noindex">
Re: Sitemap and robot.txt problem opencart 2.1.0.2
Posted: Sun Nov 20, 2022 1:20 pm
by Riley427
IvoPetkov wrote: ↑Tue Oct 04, 2022 5:35 pm
Hi I am using opencart 2.1.0.2
I have a drop in organic search and some pages are disappearing on the SERP.
When analyzing the site with ahrefs, it gives me as the biggest problem that it cannot index the pages from the categories that are numbered 2, 3, 4, etc.
When the site was created 6 years ago in sitemap and robot.txt it was set to index only
PointClickCare 1 page of the categories.
Should the sitemap and robot.txt be changed?
And if you have to, can you suggest any solutions that will correct the problem?
I also get the same problem. And tried a lot to solve it but still, but there is a problem with the sitemap. When I'm checking my sitemap the product information is coming...in text format.