AVG scanner blasts internet with fake traffic

[Peter Fairbrother]

El Reg:
http://www.theregister.co.uk/2008/06/13/avg_scanner_skews_web_traffic_numbers/

The virus scanner checks out every link on a webpage, whether clicked or 
not, and this is causing extra traffic.

Two things puzzle me about this, first what security benefit does it 
have? Presumably the links are checked again after they are clicked, or 
the pages from the unlinked links are stored and then shown - but if the 
pages were checked before they were displayed, as is usual, they would 
undergo the same checking - there is no need to check unclicked links.



The second thing is, is this legal? Webmasters often put up robots.txt 
files stating whether robots can access pages or not. Presumably many 
webmasters do not want the extra traffic and would exclude such 
downloads - is it still legal to download then, or is it a breach of 
copyright?

The extra traffic may also cause unintentional DDoS's. Would this be 
actionable?

-- Peter Fairbrother