reference > in this picture
first of all, a collection of
second method, parameter settings are grasping method noble baby, can also be carried out, as shown in figure
from the two map to see the collection of different, but the original author did not intercept love Shanghai weight flow valuation, otherwise can better compare, in fact, yesterday 14 when love Shanghai included and chain has been in the ventilation state, many sites have reduced, cannot deny the huge collection of wheat bags from a draft; but we observe mbaobao the overall structure of the website, but that can not only pay attention to the WWW domain, and more penetrated the two level domain is the need to study, such as list, item, Dudu, MKT, FY, and so on some columns of the two level domain formed a large mbaobao system, then look at these huge included above
second, about URL
the original paragraph above:
A5 search page of an article "professional: see only a small part of the wheat bags Shanghai Longfeng simple, actually yesterday to see this article, just simply sweep, without reading Cheng Fangyuan ah, today good under this article, except the first brand and fourth point flow. The other two are not quite in favor, so only the analysis of this article" barge wheat bags simple Shanghai dragon three views, a lot of communication is everything.
in the picture, the contrast between the two sites to demonstrate these differences, but only see the surface phenomenon, if we need to deal with the case, there may be two ways, the first is the same as mbaobao production, we look directly at the list.mbaobao robots.txt set above
can directly see the collected wheat bags group, 53W and 1W in direct contrast to believe that it is more admire it, these are the research system of great value, unlike the original only mentioned the WWW domain name prejudice.
in the picture, understand list two domain name has been set to shield? Dynamic web page, not let love Shanghai crawl, then love is not only to capture Shanghai with domain name?, these is to think of A.