Results 1 to 9 of 9

Thread: What is robots.txt file

  1. #1
    Join Date
    Jun 2007
    Location
    Pakistan
    Posts
    9
    Rep Power
    0

    What is robots.txt file

    Any one tell me what is robots.txt file and what is its purpose. I have uploaded it in my website totalmaza.com but I dont know what is it thats why I keep it blank, Dear friends, please any one let me know about robots.txt file.

  2. #2
    Join Date
    Jan 2007
    Location
    -
    Posts
    365
    Rep Power
    163
    It is used to control what parts or pages on your website which you don't want search engines to visit and index. Google robots.txt for more details.

  3. #3
    Join Date
    Mar 2008
    Location
    colorado
    Posts
    7
    Rep Power
    0
    You also post the path to your Google Sitemap in the robot.txt file. The google sitemap tell Google where every page in your website is located so Google knows where to search them all.

  4. #4
    Join Date
    Sep 2006
    Location
    in KL again - Cheras
    Posts
    423
    Rep Power
    166
    this is very useful if you know how to use the robots.txt to make money!

  5. #5
    Join Date
    Mar 2008
    Location
    pd
    Posts
    20
    Rep Power
    0
    How can we read the content inside?

  6. #6
    Join Date
    Sep 2005
    Location
    in my body lar...
    Posts
    1,325
    Rep Power
    0

  7. #7
    Join Date
    Mar 2008
    Location
    China
    Posts
    14
    Rep Power
    0
    thanks for sharing

  8. #8
    Join Date
    Apr 2008
    Location
    Behind Ya!!
    Posts
    49
    Rep Power
    0
    The robots.txt file is a good way to prevent this page from getting indexed. However, not every site can use it. The only robots.txt file that the spiders will read is the one at the top html directory of your server. This means you can only use it if you run your own domain.

  9. #9
    Join Date
    Apr 2008
    Location
    KL
    Posts
    21
    Rep Power
    0
    wow, cool thing! im gonna to apply it! thanks a lot!

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. What is Robots.txt
    By tedsewely in forum Search Engine Marketing
    Replies: 19
    Last Post: 18-04-2014, 02:27 PM
  2. Replies: 2
    Last Post: 04-06-2007, 08:38 PM
  3. Robots.txt
    By masrule in forum Other Internet Marketing Methods
    Replies: 4
    Last Post: 09-06-2005, 06:18 PM
  4. web log file?
    By dyda in forum Website Programming
    Replies: 3
    Last Post: 21-03-2004, 09:42 PM
  5. Replies: 7
    Last Post: 12-08-2002, 11:30 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Search Engine Optimization by vBSEO 3.5.0 RC1 PL1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112