AVG scanner blasts internet with fake traffic

[Chris Salter]

Hello Peter and UKCrypto,

Sunday, June 15, 2008, 5:10:31 PM, you wrote:


> El Reg:
> http://www.theregister.co.uk/2008/06/13/avg_scanner_skews_web_traffic_numbers/

> The virus scanner checks out every link on a webpage, whether clicked or
> not, and this is causing extra traffic.

I don't use AVG but have been using Exploit Prevention Labs
Linkscanner Pro since late 2006 (my comments are based on experience
with the latter). AVG recently acquired Exploit Prevention Labs, Inc
in order to incorporate the technology into their existing AV
products. As I understand it, Exploit Prevention Labs products also
remain available as separate products.

AFAIA Linkscanner Pro does not check out "every link on a web page". I
think the Register article refers to the *optional* facility to
precheck links on a search engine results page.


> Two things puzzle me about this, first what security benefit does it 
> have? Presumably the links are checked again after they are clicked, or
> the pages from the unlinked links are stored and then shown - but if the
> pages were checked before they were displayed, as is usual, they would
> undergo the same checking - there is no need to check unclicked links.

To explain how Linkscanner Pro works (and presumably whatever code has
been incorporated into AVG products) the following paragraphs are
lifted from a 2006 XPL white paper:

Begin Quote

SocketShield is a small utility that focuses on the stream in port 80
- the channel that connects personal computers to the web. While this
is also the channel software firewalls focus on, SocketShield does not
use the rigid rules-based non-dynamic traffic inspection typical of
firewalls. XPL's technology is highly dynamic, enabling it to react
quickly while retaining the proven reliability of signature-based
malware detection.

SocketShield harnesses LSP technology to monitor the incoming stream
of traffic for both known bad IP addresses (which are blocked
entirely) and known bad exploits. It then inspects the traffic for
known exploits, decides how to best handle the stream then acts on
that decision. SocketShield accomplishes this without impacting system
performance.

Since it operates at the Winsock level, SocketShield protects the
computer at a very low level. It must be noted, however, that
operating at this level is not without potential problems. There are
risks in interoperability, system performance and "cooperation" with
other security software. Operating at this layer is not an easy
process and specific code - a key element of SocketShield's 'secret
sauce' - has been written to handle these situations. The architecture
is such that it is able to quickly adapt for new kinds of logic
without breaking.

End Quote from
http://www.explabs.com/media/pdf/wp_xpl_foundation.pdf

What is the benefit of pre-checking a page of search engine results? I
would guess that it is intended to save time trawling through the
results (as the checks progress each result is annotated with a
graphic, e.g. a tick if the page is clean). As you visually scan the
list of results, you don't even bother to examine any result that
doesn't have a tick against it. The downside is that these checks take
time and as, at the time of writing, Linkscanner Pro 'search engine
integration' does not function with Opera, I tend to do most searches
via Opera. I have left the integration option configured so the
pre-checks work in Firefox.

As I think about it, there is one other possible 'community' benefit,
that of potentially popular (i.e. high search engine ranking) but
infected pages being identified sooner.

Is there "no need to check unclicked links"? I don't know whether
Linkscanner does check them again, but given the possibility page
being infected between the time between the first search engine
results scan and subsequent access (could be several minutes or even
longer) it would be more secure to repeat the check. That is of course
for a page that was initially clean.

> The second thing is, is this legal? Webmasters often put up robots.txt
> files stating whether robots can access pages or not. Presumably many 
> webmasters do not want the extra traffic and would exclude such 
> downloads - is it still legal to download then, or is it a breach of 
> copyright?

All LinkScanner Pro is doing is pre-checking the links on a single
search engine results page, probably ten links. I suspect that this
has only become a 'problem' given the increased number of
installations resulting from integration with AVG products. I am not
an expert but I very much doubt that anything illegal is taking place.
If I want to pre-check ten links which I may or may not subsequently
access, what law says I can't? In pre-broadband days, there were
plenty of 'web response time enhancement packages' which would
pre-fetch pages which you might or might not subsequently view. Was
that illegal?

> The extra traffic may also cause unintentional DDoS's. Would this be 
> actionable?

Again I am not an expert, but I cannot see either scenario
(unintentional DDoS's or legal action) a likely possibility.

-- 
 Chris Salter                      mailto:ukcrypto@originalthinktank.org.uk
 Cornwall United Kingdom        http://www.originalthinktank.org.uk/