Solved Downloading and and Provide an local apt-mirror

Solved issue
tinfoil-hat

tinfoil-hat

Active Member
Joined
Oct 24, 2021
Messages
440
Reaction score
220
Credits
3,054
Hi there,

like the headline says. I want to make an offline apt-mirror and also provide it via Apache2 there seem to be several possible ways on how to achive it (rsync, apt-mirror ...). Also one could get trigger a blacklist from an mirror like I learned from the past. I never understood how the hierarcy with apt-mirror works.

maybe you can help me on how to properly download a mirror and walk me then to providing the mirror (/var/spool/.. / /etc/apt/sources.list)?
 


tinfoil-hat said:
sorry, but I dont understand these instructions :(
Click to expand...
Yes, it did seem quite involved, especially with focus on the mirror aspect. When I created a local repo some years ago, it was a much simpler arrangement which avoided a lot of the details evidently now applied.

Maybe this example with clearer steps is more useful to you:
linuxopsys.com

How to Create Your Own Repository for Packages on Debian

Learn how to easily create a local Debian package repository, compatible with Debian and Ubuntu versions.
linuxopsys.com
 
osprey said:
Yes, it did seem quite involved, especially with focus on the mirror aspect. When I created a local repo some years ago, it was a much simpler arrangement which avoided a lot of the details evidently now applied.

Maybe this example with clearer steps is more useful to you:
linuxopsys.com

How to Create Your Own Repository for Packages on Debian

Learn how to easily create a local Debian package repository, compatible with Debian and Ubuntu versions.
linuxopsys.com
Click to expand...
well, with this tutorial, it's not an 1:1 copy of a debian mirror. I'd like to be able to install all debian packages from a local apt mirror without using the internet. Neither I want to create a repo with new packages.

I'd like to make the following repos for offline usage:

Code: 
deb http://deb.debian.org/debian bullseye main contrib
deb http://deb.debian.org/debian bullseye-updates main contrib
deb http://security.debian.org bullseye-security main contrib
 
tinfoil-hat said:
well, with this tutorial, it's not an 1:1 copy of a debian mirror. I'd like to be able to install all debian packages from a local apt mirror without using the internet. Neither I want to create a repo with new packages.

I'd like to make the following repos for offline usage:

Code: 
deb http://deb.debian.org/debian bullseye main contrib
deb http://deb.debian.org/debian bullseye-updates main contrib
deb http://security.debian.org bullseye-security main contrib
Click to expand...
I guess, to achieve an equivalence between a local repo and the debian ones, the creation of the local repo, as described in the link, would need to be populated by all the debian packages from the debian mirrors. That would be a significant undertaking when there are over 65,000 packages.

Otherwise I guess you have to basically mirror a mirror on your system and enter into the methods of doing that so that apt will work with all its capabilities if that is your aim.
 
osprey said:
Otherwise I guess you have to basically mirror a mirror on your system and enter into the methods of doing that so that apt will work with all its capabilities if that is your aim.
Click to expand...
that's what I want to do. But I don't know how
 
tinfoil-hat said:
that's what I want to do. But I don't know how
Click to expand...
I do admire your ambition. I can only refer you back to the first link in post #2. Essentially it seems to me that to create a full mirror, you would need look into the ftpsync script and go from there. The text specifically warns:

Do not use your own scripts, and do not just use single-pass rsyncs.
Click to expand...

Your mirror would normally need to be updated, so the text goes on to say:

Using ftpsync ensures updates are done in a way so that apt does not get confused. In particular, ftpsync processes translations, contents, and other metadata files in an order such that apt is not running into validation errors should a user update the package list while a mirror-run is in progress. Furthermore, it also produces trace files that contain more information that is useful for determining whether a mirror works, what architectures it carries, and where it syncs from.
Click to expand...

I guess that wanting the mirror to be offline, still needs ftpsync to get it all up in the first place.

Managing the complexity has been built into the ftpsync script, and that's because the steps involved in the process are involved, so doing it manually in many separate steps is likely prone to error and failure.

If you want something less than equivalence with a mirror for your own site, the text proposes the package : debmirror.

If you want to operate a mirror for your own site only, and you need to carry only a subset of suites (such as stable), debmirror might be a good fit for you also.
Click to expand...

I wouldn't underestimate the learning curve in this project.
 
Last edited:
osprey said:
I do admire your ambition. I can only refer you back to the first link in post #2. Essentially it seems to me that to create a full mirror, you would need look into the ftpsync script and go from there. The text specifically warns:



Your mirror would normally need to be updated, so the text goes on to say:



I guess that wanting the mirror to be offline, still needs ftpsync to get it all up in the first place.

Managing the complexity has been built into the ftpsync script, and that's because the steps involved in the process are involved, so doing it manually in many separate steps is likely prone to error and failure.

If you want something less than equivalence with a mirror for your own site, the text proposes the package : debmirror.



I wouldn't underestimate the learning curve in this project.
Click to expand...
I feel like an idiot, normally I search up my problems and work some tutorials trough. Bit ftpmirror is very badly documented... I'd very much apreciate if you'd spoonfeed me on that. I really have no clue on how to use ftpmirror. I am so close to smash my keyboard ...
 
you could as well tell me: here are the parts, now build me an engine
 
tinfoil-hat said:
I feel like an idiot, normally I search up my problems and work some tutorials trough. Bit ftpmirror is very badly documented... I'd very much apreciate if you'd spoonfeed me on that. I really have no clue on how to use ftpmirror. I am so close to smash my keyboard ...
Click to expand...
No need to feel that way ... it's rather more of a brave undertaking.

Perhaps have a look here to check out the sort of steps involved. It's from 2013, but looks like it still covers the general process: http://www.linux-admins.net/2013/11/creating-official-debian-mirror-with.html.

Bear in mind that some things may have changed since that time.
 
Well, chat-gpt did the trick, I made a wiki article about it

Create a local apt-mirror | BookStack

Download an apt-mirror To create a local Debian APT mirror, you can follow these general steps: Se...
wiki.tinfoil-hat.net wiki.tinfoil-hat.net
 
Last edited:
Inspired by your efforts, I asked bard how to create a local mirror, and here is the answer:
Code: 
how to make a local apt repository for debian
View other drafts

Sure, here are the steps on how to make a local apt repository for Debian:

    Install the dpkg-scanpackages tool.

Code snippet

sudo apt-get install dpkg-dev

Use code with caution. Learn more

    Create a repository directory and place packages there.

Code snippet

mkdir /path/to/repository
cp /path/to/packages /path/to/repository

Use code with caution. Learn more

    Run dpkg-scanpackages.

Code snippet

cd /path/to/repository
dpkg-scanpackages -m . > Packages

Use code with caution. Learn more

    Create a Debian configuration file for the repository.

Code snippet

vi /etc/apt/sources.list.d/local.list

Use code with caution. Learn more

In the configuration file, add the following line:
Code snippet

deb [trusted=yes] file:/path/to/repository ./

Use code with caution. Learn more

    Update your apt repository.

Code snippet

sudo apt-get update

Use code with caution. Learn more

Now, you can install packages from your local repository by using the apt-get command. For example, to install the mydeb package, you would run the following command:
Code snippet

sudo apt-get install mydeb

Use code with caution. Learn more

Here are some additional notes:

    The trusted=yes option in the configuration file tells apt that the local repository is trusted. This means that apt will not prompt you to confirm the installation of packages from the local repository.
    You can use the dpkg-scanpackages tool to update the Packages file whenever you add or remove packages from the repository directory.
    You can also use the apt-ftparchive tool to create a more comprehensive local repository that includes metadata about the packages, such as their descriptions and dependencies.

I hope this helps! Let me know if you have any other questions.

It's untested by me, and my own experience with such mirrors isn't recent enough to evaluate it.
I guess it's like a lot in linux, many means to make margarine.
 
Last edited:
You must log in or register to reply here.

Members online

Total: 492 (members: 4, guests: 488)

Latest posts

Top