sofasurfer
Active Member
I'm using Ubuntu 20.04
Is there a way, preferably from the command line, to find the total size of a website?
Is there a way, preferably from the command line, to find the total size of a website?
Can I find the size of an entire website?
- Thread starter sofasurfer
- Start date
Not realistically, no...
Unless, of course, you're hosting the website. If you're hosting the site, just use 'ncdu' on the appropriate folder.
If you do not host the site, then no... No, you can probably use httrack/wget to download all the pages (considered kinda rude), but that'd be just the pages and images and stuff like that... There's often still a lot you won't see and download, like the database that populates all that.
So, no... That's not something you can realistically do.
Unless, of course, you're hosting the website. If you're hosting the site, just use 'ncdu' on the appropriate folder.
If you do not host the site, then no... No, you can probably use httrack/wget to download all the pages (considered kinda rude), but that'd be just the pages and images and stuff like that... There's often still a lot you won't see and download, like the database that populates all that.
So, no... That's not something you can realistically do.
Puppy Linux has a software --
Pfetch allows downloading an entire site, from which should allow one be able to determine size.
Another link from cyberciti
Pfetch allows downloading an entire site, from which should allow one be able to determine size.
Another link from cyberciti
That is only true if the site is pure HTML type stuff. You won't get things like the backend stuff - databases and stuff like that. My linux-tips site is about 10 GB in size, with probably about 1/20 of that being accessible publicly with tools like Pfetch or HTTrack (or wget, which you can set to no-clobber at least).
you guys getting all fancy with the site downloaders .. back in the day we just used 
That's how I built most of this site: https://web.archive.org/web/19981205165509/http://linux-howto.com/
That's from 98 .. used wget to mirror all of the LDP and some other stuff nightly and convert it to html.
wget --mirror That's how I built most of this site: https://web.archive.org/web/19981205165509/http://linux-howto.com/
That's from 98 .. used wget to mirror all of the LDP and some other stuff nightly and convert it to html.
Last edited:
HTTrack Website Copier - Free Software Offline Browser (GNU GPL)
HTTrack is a free (GPL, libre/free software) and easy-to-use offline browser utility. It allows you to download a World Wide Web site from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer. HTTrack...
Code:
httrack.x86_64 : Website copier and offline browser
