New to the community? Start here

[Snapshot Pro] Snapshot Issues

I’m having an issue with a few sites where Snapshot is storing backup files taking up space on my server when they should just be being stored on the WPMUDEV cloud. Can you help me work out why.

Tue, 3 Mar 2020 12:40:07
More Snapshot Pro discussions
Snapshot Pro
  • Adam Czajczyk
    • Support Gorilla

    Hi Jon

    I hope you’re well today and thank you for your question!

    The Snapshot plugin has two built-in backup tools: snapshots (as in “Snapshot -> Snapshots”:wink: and “Managed Backups” (as in “Snapshot -> managed backups”:wink:.

    The first one will always store at least one backup locally while the second one should be removing local backup files right after they are successfully uploaded to The Hub.

    From your question, I understand that you’re running Managed Backups. If those seem to complete successfully and yet, there are still backup files stored locally that would mean one of two things:

    – either there are also snapshots running (“Snapshot -> Snapshots”:wink: and if so, the only thing that could be done would be to minimize the space it takes by making it keep less backups; that can be set in each of the snapshots configuration (“Snapshot -> Snapshots -> “Edit” for every defined snapshot) in “Local storage limit” section that is visible if snapshot is set to run on schedule (it’s not available if it’s ‘once off” snapshot)

    – or the managed backup actually fails – either to be completed or to be uploaded to The Hub.

    That said:

    – are you running only Managed Backups or both Managed Backups and snapshots?
    – did you check if backups are successfully uploaded to The Hub ?

    Let me know please. It would also be helpful if you could enable support access to (for the start) one of the affected sites. To do so, go to the “WPMU DEV -> Support -> Support Access” apge in site’s back-end, click on “Grant support access” button there and let me know once it’s done.

    Kind regards,
    Adam

  • Jon
    • The Crimson Coder

    I’m a little unsure. I think I’m just using managed backups (not snapshots) and it looks like they’re uploading to the hub fine but there are large files still sitting on the server which I don’t think they should.

    I’ve granted support access for forresholidays.org – are you able to take a look and help me work it out. It’s happening on two other sites which are linked to this but completely separate installs but may be let’s start with this?

  • Predrag Dubajic
    • Support

    Hi Jon,

    Thanks for granting access, I had a look at your site and can see that the backups are properly created so they shouldn’t be on your server anymore.
    I did notice that there was one backup that was taken 2 weeks ago but was uploaded just yesterday, so it could be that there was some hiccup at that specific time that prevented the communication with your site and the upload failed so the plugin stayed on your server.

    Is this the only backup not removed from your server or are there more files left there?

    Could you provide us with FTP or cPanel access to this site so we could check further why the backups aren’t getting removed?
    Note: Don’t leave your login details in this ticket.
    Instead, you can send us your details using our contact form https://wpmudev.com/contact/#i-have-a-different-question and the template below:
    Don’t change the selected topic in the dropdown, just leave it at “I have a different question”.

    Subject: "<strong>Attn: Predrag Dubajic</strong>"
- Admin login
Login URL:
Admin Username:
Admin Password:
---
- FTP credentials
Hostname:
Username:
Password:
Port:
Key-File ( and password ) if needed
---
- Server Admin ( CPanel / Plesk )
Login URL:
Username:
Password:
---
- Folder path to site in question
- Link back to this thread for reference
- Any other relevant urls/info

    Best regards,
    Predrag

  • Panos
    • SLS

    Hi Jon ,

    Those zip files are leftovers from backups that did not manage to complete due to timeouts caused by system running out of resources. In short, the zip files got too big for the server to handle. This seems to be not consistent, so it can be related to site traffic at the time, specific processes running in the same time with backup, other cron jobs, if it is on shared hosting other shared sites’ load etc.

    The ones in the snapshots/_backup folder are from several attempts to complete the same backup and each time it would time out. The zip in the snapshots folder is the one left probably when the zip was created from the ones in the _backup folder but it either timed out before the zip was completed or during upload.

    In any case this is related to resources, as handling zip files of 500mb is resource intensive. You can delete those files so they don’t take up your space and with your permission I could run a few tests to see if there is anything we can do in order to reduce this high load, by using some defines in Snapshot. It’s a long shot (as it’s not consistent) and we would need to do multiple tests with those in order to see if those would indeed help in your case, that’s why I would need your confirmation first.

    Also I would like to inform you that our devs are currently re-writing Snapshot in order to support incremental backups using much less resources. Still in beta though ( https://wpmudev.com/forums/topic/snapshot-4-0-new-engine-and-incremental-backups/ ) so I can’t give any ETA on it.

    Kind regards!

  • Panos
    • SLS

    Hi Jon ,

    I wanted to try adding some the following definitions in the wp-config.php file:

    define('SNAPSHOT_ATTEMPT_SYSTEM_BACKUP', true);
define('SNAPSHOT_MB_BREADTH_FIRST', true);
define('SNAPSHOT_TABLESET_CHUNK_SIZE', 100);
define('SNAPSHOT_FILESET_CHUNK_SIZE', 100);
define('SNAPSHOT_FILESET_LARGE_FILE_SIZE', 104857600);

    but I noticed that the ftp access is no longer working.

    You can try adding those on your own and keep an eye on the zip files there.

    Something else that might help reduce resource demand, could be to exclude the plugins and theme’s folders, especially if you have a big number of plugins. Those can be considered as static content, meaning that those can be reinstalled, and focus backups on db and media files.

    I can still add these definitions if you like. In such case can you re-send ftp access as mentioned here : https://wpmudev.com/forums/topic/snapshot-pro-snapshot-issues/#post-3732394. You can set the subject though to Attn: Panos Lyrakis

    Kind regards!

  • Panos
    • SLS

    Hi Jon !

    It actually is the same with the one I already had. I tried it again just in case also asked Adam to try too, but non of us managed to connect.

    Could you resend in ftp information by including all info, host, username and password? Could you also try it on your side to make sure those are valid?

    One more thing that might affect, have you any country blocking active? In such case I could use a vpn and give it a try.

    Thanks!

  • Panos
    • SLS

    Thanks for sending those in Jon !

    I see that latest backups are not completed so it’s definitely related to server resources. I have excluded the plugins and themes from the backups also added the defines mentioned above in your wp-config.php file.

    I couldn’t wait for a scheduled backup to start so I ran a new managed backup and it completed successfully. This should also help the daily scheduled backups too.

    You can keep an eye in your backups page in admin in the following days:
    site.org/wp-admin/admin.php?page=snapshot_pro_managed_backups
    where it should list your daily backups.

    Kind regards!

  • Jon
    • The Crimson Coder

    There are still some big ZIP files in that folder that are taking up server space. That was the original issue – backups were happening (to WPMUDEV servers) but weren’t being removed from the server so I was regularly hitting my quota.

  • Panos
    • SLS

    Those are the ones that failed to complete and that’s why they were not removed. You can delete the ones that are there manually from cpanel (it will take much longer via ftp). So my next concern was to avoid repeating this issue again and keep adding new leftovers when future backups fail to complete.

    With thosee changes mentioned above I believe that they won’t be failing so there shouldn’t be new huge zips added in your uploads folder.

    Hope this clarifies why I wanted to do the above steps :slight_smile:

  • Panos
    • SLS

    You can try adding the following defines in the wp-config.php file

    define('SNAPSHOT_ATTEMPT_SYSTEM_BACKUP', true);
define('SNAPSHOT_TABLESET_CHUNK_SIZE', 100);
define('SNAPSHOT_FILESET_CHUNK_SIZE', 100);
define('SNAPSHOT_FILESET_LARGE_FILE_SIZE', 104857600);

    Make sure you place those before this line
    /* That's all, stop editing! Happy blogging. */

    Next you can try excluding plugins and themes from backups as done in the current site. To do so, you can add the paths to those folders in Snapshot settings under the Global File Exclusions section here : yoursite.com/wp-admin/admin.php?page=snapshot_pro_settings

    In order to get the full paths for those folder you can go to Tools > Site Health > Info. Those should be in the Directories and Sizes accordion:
    yoursite.com/wp-admin/site-health.php?tab=debug

    You can instead use :

    /plugins/
/themes/

    for those instead of full-path.

  • Panos
    • SLS

    Hi Jon ,

    I just tried now and the backup completed. As this is a backup written in php, it does add a lot of overhead on the server on it’s own (that’s why we’re re-writing it so it supports incremental backups ), but there are other facts that affect the backup too. For instance other processes that might be running at the same time on server, number of visitors, if it is on a shared environment number of visitors on other sites that share resources and so on.

    Something else that might be happening is that when running a managed backup manually, this is done via ajax. So if the pc/laptop you are using goes idle, then the process will fail. When testing that make sure you set your pc/laptop to not “sleep” for a few hours, until this finishes.

    For the other sites you mentioned could you create a separate task so we can give them a look too? Or you can use our chat support and provide admin/ftp access there directly.

    Kind regards!