Nutch's robots cache

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Nutch's robots cache

Brian Whitman
Is there a way to programatically get at Nutch's robots.txt cache  
after a fetch? I have other non-nutch fetchers that operate on the  
same URLs and it'd be silly to waste the accesses.