Shared Left Border




Help Yourself
And Others


What's New on
AYP Plus?

AYP Retreats
Worldwide Listing

AYP Teacher Training

Local Contacts
Meditation Groups,
Training & Retreats

Key L

Start Lessons Here

Desire - Bhakti

Deep Meditation


Spinal Breathing



Sambhavi Mudra

Asanas - Postures


Yoni Mudra

Uddiyana - Nauli

Navi Kriya

Kechari Mudra

Chin Pump

Eight Limbs of Yoga

Samyama & Siddhi

Spinal Bastrika

Heart Breathing

Yoga Nidra
Cosmic Samyama

Mantra Enhancements

Solar Centering



Vajroli Mudra

Whole Body Mudra

Yogic Diet






Nectar Cycle


Sensitivity Issues

Spirits & Entities


Jnana - Advaita

End of Suffering

Karma Yoga

Dare to Dream

Evolution of Action

Prayer & Healing

Yoga for Children


Summary of Practices

Time Management

Modifications to
Baseline System





More Resources

AYP Plus

Public Forum

Plus Forum

AYP Survey

audioicon1.jpg (812 bytes)
AYP on Radio & TV

Yoga FAQ

Sanskrit Glossary

AYP Books & Audio


AYP Books Flyer

Extended Booklist


Press Releases

Articles & Media


Website Traffic


AYP-Related Websites

Social Networks


AYP Public Forum
AYP Public Forum
AYP Home | Main Lessons | Tantra Lessons | AYP Plus | Retreats | AYP Books
Profile | Register | Active Topics | Members | Forum FAQ | Search
Save Password
Forgot your Password?

 All Forums Forum
 AYP Helpers
 robots.txt for the AYP website
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  


20 Posts

Posted - Mar 23 2014 :  12:40:09 AM  Show Profile  Reply with Quote  Get a Link to this Message
Hi I noticed that when using Google to search for certain topics the AYP website returns pages titled "AYP Support Forums - Advanced Yoga Practices" all of which have the URLs in the following format:

I think it would be better to configure the robots.txt file so that instead of the above the following is displayed:

For example take a look here:



5192 Posts

Posted - Mar 23 2014 :  09:55:00 AM  Show Profile  Visit yogani's Homepage  Reply with Quote  Get a Link to this Reply
Hi YM:

So what you are saying is that certain Google searches are going to the print pages, rather than to the actual forum topics? (Fortunately, the print pages include links to the actual topics.) and the forums are site mapped with code set up by our tech consultant years ago. As far as I know, the site mapping is working, because the actual forum topics receive a lot of traffic from Google.

I will have our tech consultant look into the anomaly you mentioned to see what is going on. It could be that certain public pages, like the print pages, will come up with particular search term configurations like you tried. But not all searches will be like that, and hopefully most are going to the actual forum topics. I don't know that we could account for every variation in search terms that might lead to public pages in the forum that are not actual topics. There are thousands of search term variations and thousands of forum pages. But we will take a look.

Thanks for pointing this out.

The guru is in you.

PS: Whether a robots.txt file is a solution or not is for our tech consultant to decide. I only know how to use it for blocking search engines, and we are certainly not going to do that.

Go to Top of Page


5192 Posts

Posted - Mar 23 2014 :  2:59:08 PM  Show Profile  Visit yogani's Homepage  Reply with Quote  Get a Link to this Reply
Hi YM:

The feedback from our tech consultant is that it is not possible to predict what Google ranking algorithms will produce, and second guessing them could be counterproductive. Who knows why Google may sometimes rank a print page above the actual page? (at least we have the real link on the print page) We don't want to block those print pages from Google, because it could adversely affect the visibility of related pages.

You did not mention how widespread this issue is. If it involves hundreds or thousands of pages, we can take a closer look. But if it is affecting only a few pages, then it is probably best to leave it alone.


The guru is in you.

Go to Top of Page


20 Posts

Posted - Mar 31 2014 :  11:17:32 PM  Show Profile  Reply with Quote  Get a Link to this Reply
Fair enough I guess. Currently there's no robots.txt on the AYP site as far as I can tell:

Preventing print pages from being indexed is easy enough - just create a text file at the above location and add something like this:

User-agent: *
Disallow: /pop_printer_friendly.asp

I don't think doing this will have negative consequences in terms of ranking because the main pages beginning with /topic.asp have the same content. Seems that searching for minor topics on Google usually produces this kind of "anomaly":

Anyway it's obviously up to you and the tech consultant to decide what to do with it - just wanted to report the issue because I was experiencing it every other time I did site search on Google.
Go to Top of Page


5192 Posts

Posted - Apr 01 2014 :  10:23:54 AM  Show Profile  Visit yogani's Homepage  Reply with Quote  Get a Link to this Reply
Hi YM:

The question is, if blocked, where will those "searches for minor topics" go? If not to AYP, then it would be a net loss of traffic.

Correct, there is no robots.txt file on . We do have one blocking the entire site from the search engines, because mirror sites are not permitted by Google, and both would be blocked by them if one was not blocked by us. Years ago, was chosen as the search site, because it includes both the lessons and the forums, whereas is the lessons only. As mentioned before, (including the forum) is site mapped for improved search engine exposure.

The thing with doing search engine access modifications is that no one knows what the Google algorithms are, and there is risk in it. We could end up shooting ourselves in the foot for the sake of a few errant search results, which have links to the corresponding topics anyway. Very often, "upgrades" result in unexpected downsides due to unknown factors in the technology.

So from the perspective of a long time, well-established search engine exposure program, this is a case of, "If it ain't broke, don't fix it." At least not for now...

Thanks for poking around. We are always looking for ways to improve the AYP resources and their visibility. But at this stage we are not inclined to do modifications that offer a small benefit coupled with a potentially big downside risk.

The guru is in you.

Go to Top of Page


295 Posts

Posted - Apr 01 2014 :  10:46:18 AM  Show Profile  Reply with Quote  Get a Link to this Reply
Most likely what you want to do is to redirect these links to the main pages, not block them. This can be done with what are called "regular expressions" to grab all pages that match a pattern in the URL structure. Redirecting pages is common, as people move content and rename pages (and entire websites). I've done this plenty of times with my business site which relies heavily on ranking high in Google results, and have never seen a negative impact.

Your comment regarding duplicate content is correct, but keep in mind that having both a webpage and the print page show up in search results might be seen by Google as duplicate content (which it is).

That said, I also agree with "if it ain't broke don't fix it". But as Wayne Gretsky said, you miss 100% of the shots you don't take :)

The question is whether Google penalizes for this, and it is probably one area they do reveal their stance on. The other question is if it is a pain for new visitors, does it turn them off from exploring AYP's website?
Go to Top of Page


295 Posts

Posted - Apr 01 2014 :  10:51:02 AM  Show Profile  Reply with Quote  Get a Link to this Reply
Here's some great info on the topic, and some associated solutions:

It seems the rel=canonical solution is the ideal one here.

Edited by - Yogaman on Apr 01 2014 10:56:37 AM
Go to Top of Page


5192 Posts

Posted - Apr 02 2014 :  11:43:06 AM  Show Profile  Visit yogani's Homepage  Reply with Quote  Get a Link to this Reply
Hi YM and YM:

We will look into this some more. We don't want multiple results for forum pages turning up often in Google, as it could reduce exposure.



Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
AYP Public Forum © Contributing Authors (opinions and advice belong to the respective authors) Go To Top Of Page
This page was generated in 0.05 seconds. Snitz Forums 2000




Join the Mail List:
News and Events
Near You


Direct Downloads

eBooks - PDF, EPUB

FREE eBooks with
MP3 AudioBooks

SAVE  with Bundled
Multi-Title Downloads

AYP Book Translations

AYP Books

(PDF Books Flyer)

Easy Lessons

Advanced Yoga Practices Book

Adventure Novel

The Secrets of Wilder Novel

Enlightenment Series

Deep Meditation Book

Spinal Breathing Pranayama Book

Tantra Book

Asanas, Mudras & Bandhas Book

Samyama Book

Diet, Shatkarmas and Amaroli Book

Self-Inquiry Book

Bhakti and Karma Yoga Book

Eight Limbs of Yoga Book

Eight Limbs of Yoga Book

Questions & Answers



Public Home | Plus Home | Main Lessons | Tantra Lessons | Topic Paths | Interviews | Videos | Site Search | Links Section | Testimonials | AYP Survey | Books | Downloads | Contact