File robot chuẩn seo

User-agent: *
Disallow: /cart/
Disallow: /checkout/
Disallow: /buyer/login/otp
Disallow: /user/
Disallow: /me/
Disallow: /order/
Disallow: /daily_discover/
Disallow: /mall/just-for-you/
Disallow: /mall/*-cat.
Disallow: /from_same_shop/
Disallow: /you_may_also_like/
Disallow: *-i.*/similar
Disallow: /find_similar_products/
Disallow: /top_products
Disallow: /search*searchPrefill
Disallow: /index.html
Disallow: /search*hashtag=
Disallow: /search?shop=
Disallow: /buyer/*/rating/
Disallow: *-cat.*?category=
Disallow: /shop/*/search
Disallow: /shop/*/details
Disallow: /search/?
Disallow: /search?brands=
Disallow: /search?originalCategoryId= 
Disallow: /addon-deal-selection/
Disallow: /bundle-deal/
Disallow: /search_user
Disallow: /search?keyword=*.com
Disallow: /blog/?s=
Disallow: *?sp_atk
Disallow: /search?keyword=*.com
Disallow: /search?keyword=*·com
Disallow: /search?keyword=*.tv
Disallow: /search?keyword=*.xyz
Disallow: /search?keyword=*。Com
Disallow: /search?keyword=*·COM
Disallow: /search?category=*.com
Disallow: /search?category=*·com
Disallow: /search?category=*.tv
Disallow: /search?category=*.xyz
Disallow: /search?category=*。Com
Disallow: /search?category=*·COM
Disallow: *utm_source

User-agent: Googlebot
Disallow: /?add-to-cart=
Disallow: /?add_to_wishlist=
Disallow: /feed/

User-agent: Googlebot-News
Allow: /feed/
Disallow: /wp-admin/
Disallow: /readme.html
Disallow: /license.txt
Disallow: /?s=*
Allow: /wp-admin/admin-ajax.php
Allow: /wp-admin/images/*

User-agent: Googlebot-Mobile
User-agent: Bingbot
Sitemap: https://nguyenthaimmo.com/sitemap_index.xml

Giải thích:

1. User-agent: * – Quy tắc này áp dụng cho tất cả các bots.
2. User-agent: Googlebot – Các quy tắc cụ thể cho Googlebot.
3. User-agent: Googlebot-News – Các quy tắc cụ thể cho Googlebot-News.
4. User-agent: Googlebot-Mobile và User-agent: Bingbot – Định nghĩa các bot cụ thể nhưng không có quy tắc riêng (được mặc định theo quy tắc chung của *).
5. Sitemap: – Đường dẫn tới sitemap của website.