Can account backups seperate mail folder from other account information

[martin MHC]

We have regular account backups that are generated every day. These backups contain all the account information for each account and are stored as .tar.gz files.

Recently we needed to explore the SQL data in some backups and found issues that the backup file size was so big it was causing a big slowdown in processing this request (from the client).

The account.tar.gz was 24Gb - This was stored remotely and so we needed to download 24Gb in order to open the file and then to extract the account.sql which was 100kb.

Almost all of the backups size is emails. Is there any way, or any how-to guide as to creating a backup system that stores account details seperately from mail details such as:

account.tar.gz = All account files and folders EXCEPT mail (and any related mail data)
account-mail.tar.gz = the account /mail/ folder (and any related mail data)

This would help enormously in only needing to download much smaller data amounts and opening much smaller files when we simply don't need to explore clients vast emails

Cheers

 

[cPRex]

Hey there! We wouldn't have an automated tool that would let you split the backup types, but it is possible to pull out files from a tar file without extracting the entire tarball:

How to extract specific file(s) from tar.gz

How can we extract specific files from a large tar.gz file? I found the process of extracting files from a tar in this question but, when I tried the mentioned command there, I got the error: $ ta...

unix.stackexchange.com

If you have a tool like JetBackup you could use that to exclude certain directories, allowing that backup tool to make a partial backup of the account for you:

Exclude Files and Directories — JetBackup Documentation documentation

docs.jetbackup.com

 

[martin MHC]

If you have a tool like JetBackup you could use that to exclude certain directories, allowing that backup tool to make a partial backup of the account for you:

Exclude Files and Directories — JetBackup Documentation documentation

docs.jetbackup.com

This could work but would need to tell the WHM backup system to make two backups -- one of the full account excluding /mail/ and one of the /mail/ folder in isolation. Would this be possible (I am also unfamiliar with JetBackup biut it states it requires WHMCS which I'm unclear if that is the same thing as WHM or if it's another 3rd party software requirement? )

 

[cPRex]

WHMCS isn't required for the JetBackup tool to be installed for function. Can you let me know where you found that? Here is the page with the system requirements:

Installation — JetBackup Documentation documentation

docs.jetbackup.com

I was thinking one backup with cPanel tools and one with JetBackup and that would get you both versions you need.

 

[ffeingol]

Just wondering what kind of access you have to the remote server? When we need to do this we normally do something like the following

tar -tzvf account.tar.gz | egrep sql

That will then list out the full path to all the .sql dumps. You can then run

tar -xzvf account.tar.gz /full/path/to/database.sql

And it will extract only the database dump that you want. Assuming you can do this all on the remote server, you'll save your self a lot of time/effort/bandwidth.

 

[martin MHC]

Just wondering what kind of access you have to the remote server? When we need to do this we normally do something like the following

I would normally do this but have no command line access to remote files in any way, and the remote system won't open the files because they're too big so they need to be downloaded.

 

[martin MHC]

WHMCS isn't required for the JetBackup tool to be installed for function. Can you let me know where you found that? Here is the page with the system requirements:

Installation — JetBackup Documentation documentation

docs.jetbackup.com

I was thinking one backup with cPanel tools and one with JetBackup and that would get you both versions you need.

Sorry, My mistake, I was looking at the JetBackup WHMCS page . Good idea to do a single JetBackup one simply omitting the /mail/ folder.

 

[ffeingol]

You can actually do this pretty straight forward in JetBackup. When you do an 'accounts' type backup you get to choose what's being backed up. In one backup job you could only include email and in another you could have everything but email. This page in their doc has a screen shot of all the items you can include/skip in a full account backup: Create New Backup Job — JetBackup Documentation documentation

 

[mywhm]

Hey there! We wouldn't have an automated tool that would let you split the backup types, but it is possible to pull out files from a tar file without extracting the entire tarball:

tar.gz files do not have an index.
In order to show you which files are contained in the archive, tar indeed needs to uncompress the archive and extract the files

Not indexed: In order to see which files are contained in a tar archive, it is necessary to scan through the entire archive. This can be impractical if the archive is very large.

Does tar -tvf decompress the file or just list the names?

I have a tar.gz file of 32 GB. I was trying to extract specific folders from it so I listed the contents with the following command to view the files structure: tar -tvf file.tar.gz > files.txt...

askubuntu.com

duplicity: New file format

 

[cPRex]

@mywhm - while there is no index, the process I mentioned let's tar read through all the files it would extract, and lets you pull out what you want. It may not be fast, but if disk space is an issue it can help you from needing to clear up space to make room for an entire unzipping process.