Here we go again - ISP DPI, but is it interception?

[Tom Thomson]

 > Question: why don't TalkTalk instead just record the IP of visited
> addresses (as I understand they are currently required to do anyway),
> then spider those addresses in the usual way (in accordance with
> robots.txt)?

A single IP address can host many web sites; each has its own robots.txt file.  If all you have is the IP you have neither the site root URLs nor those of the robots.txt files so an IP address alone doesn't give you a start point for spidering.

M.