Today's article has you ignoring perfectly good security warnings to get what you want...

KGIII

Super Moderator
Staff member
Gold Supporter
Joined
Jul 23, 2020
Messages
11,498
Reaction score
9,993
Credits
95,326
It may look familiar as I did a similar article for cURL, which I refused to stylize as such in this article, and familiarizing yourself with the first article will help some of you understand this one better. I suspect many of y'all are already familiar with security certificates.

Well, this article tells you how to ignore certificate errors with 'wget'. It's not something you may want to do - but it's something you *can* do. Being ever so helpful, I even tell you how to configure wget to always ignore such errors. You're welcome!


Actually, I'm one of those people who thinks the entire web doesn't actually need SSL. I use one, 'cause my site collects some personal information. Also, because if I didn't the browsers throw horrible warnings as though the sky was falling. Ah well... Feedback is fine, I suppose. This was authored quickly, but is legible.
 


Good one. I didn't respond directly on the blog because I don't have anything to contribute. But it's good to know when I'm pilfering ... er, I mean ... borrowing ... libraries of data. Don't worry, it's all legal, just probably not the path the site wants me to get it.
 
Don't worry, it's all legal, just probably not the path the site wants me to get it.

Yeah, wget is pretty versatile if you're any good with it. I do ask that if you decide to scrape my site - please consider using the no-clobber flag. Well, if you don't use it the system will probably consider you a 'bad bot' and ban your IP address. But, still, please be gentle!
 
Good article. I don't very often use wget but I had never thought about the certificates issue with it. Nice.
I agree that not every site 'needs' HTTPS or other security certs.
Probably no need to mention this to anyone using wget but to someone just starting out with it - it may be worth mentioning that it could be considered rather 'rude' to download someone's whole site.
And with wget it is rather easy to forget and download a lot more than one intended to!
 
Yeah, wget is pretty versatile if you're any good with it. I do ask that if you decide to scrape my site - please consider using the no-clobber flag. Well, if you don't use it the system will probably consider you a 'bad bot' and ban your IP address. But, still, please be gentle!
Don't worry, not talking about your site.
 
it may be worth mentioning that it could be considered rather 'rude' to download someone's whole site.

I do hope that anyone scraping my site with wget is nice enough to use the no-clobber option, though I think the site may bitbucket your traffic automatically if you tried it without that flag - it'd see it as an attack. I tested it on a similarly configured site a while back and managed to ban my own IP address. Thankfully, I have a VPN available.

I agree that not every site 'needs' HTTPS or other security certs.

There's been a push for it for years and now you pretty much have to - or your visitors will get warnings in their browser. At least they can now be had for free, for the basic certificates at least. You can set up LetsEncrypt pretty easily - and then it's also easy to schedule it to automatically update before the existing certificate expires.

So, it's just a bit of knowledge people may need to add. Some control panels will do all that automatically for you, so you never see the man behind the curtain - for better or worse.

Anyhow, glad ya liked it. Obviously, there's a similar article for cURL if you're a frequent user of that.
 

Members online


Top