Here we go again - ISP DPI, but is it interception?

Tom Thomson colinthomson1 at
Sat Jul 31 15:28:56 BST 2010

 > Question: why don't TalkTalk instead just record the IP of visited
> addresses (as I understand they are currently required to do anyway),
> then spider those addresses in the usual way (in accordance with
> robots.txt)?

A single IP address can host many web sites; each has its own robots.txt file.  If all you have is the IP you have neither the site root URLs nor those of the robots.txt files so an IP address alone doesn't give you a start point for spidering.


More information about the ukcrypto mailing list