a question about preventing link rot...

[ratcreature]

With each update of my recs page I run a link checker as part of the validation, using the W3C online tools. However a good number of sites, both archives and LJs, have robot exclusion rules, so I can't readily see whether the stories in question are really still there with the W3C link checker. Initially it wasn't a huge bother to check them manually, but by now it's over sixty links. Do any of you who maintain largish recs pages or other link collections have a good strategy for this or an autochecker tool working with a website that's not excluded as a robot?

I mean something like the bookmark editors in webbrowsers that go through links and then show you the broken ones, only something that would check based on a website. I assume html editors or website tools might have such a function, but I code my page simply in Emacs with the help of html and php editing modes for highlighting and such, and I have never needed any specialized programs. I tried one html editor that was bundled with my distro and had been installed automatically as part of the standard set-up (Quanta?), but it all seemed rather more complicated than I wanted to get into for just checking a bunch of links, i.e. I couldn't get the link checker function to work properly with my file at first try. I have no idea whether that was because of the file not being plain html but php mixed with html or something else, but I decided I'd really prefer something simpler.

So how do you check your links?

Comments

[kassrachel.livejournal.com]

This is an unrelated comment -- I don't have anything to add about link-checking -- but just wanted to say Värttinä, yay! They're one of my favorite bands. Are they well-known in Germany? I find that only a few Americans recognize them, but those who do tend to be interesting people w/ interesting musical tastes. :-)

[ratcreature]

Well, I initially found them because someone on my f-list shared some songs, and then I looked for their CDs, so I couldn't really say how widely known they are here. I mean, I haven't heard them on the radio, and most of my friends aren't really into folk. However considering that I found a largish selection of their CDs in the first mainstream music store I looked they can't be really obscure here. I mean, the time before that when I tried finding CDs with music shared on my f-list I had to resort to some obscure specialist importer of independent music labels, pay an outrageous price and wait for weeks, because no music store nor amazon sold the CD.