Skip to main content
Code42 Support

Archive Maintenance

Applies to:
  • CrashPlan PROe

Overview

Archive maintenance is a regularly-scheduled task that runs on each backup destination. The purpose of archive maintenance is to maintain archive integrity and to optimize the size of the archives by:

  • Checking backup archives for corrupted files and repairing any detected corruption
  • Pruning file versions and removing deleted files according to the frequency and version settings
  • Purging files that are no longer selected for backup

All backup destination types (cloud, computer, and folder) perform archive maintenance.

Archive maintenance in depth

The major operations that archive maintenance performs are defined and described below.

Pruning

In CrashPlan PROe, pruning is the process that removes files and versions as defined by the frequency and version settings.

pruning

Item Description
1 Block data files (BDFs) Files stored in the archive as blocks of data.
2 Unused block data files Pruning marks blocks as unused. Think of these blocks as "holes" that still take up space in the BDF until the compacting process completes.

Compacting

Compacting is the process that frees up space on the store point's file system after pruning.

compacting

Item Description
1 Unused block data files (BDFs) Blocks marked as unused by the pruning process.
2 New block data file The storage server rewrites the relevant BDF and omits the "holes" described in the previous section. The new BDF is smaller than the original.

Validation and healing

Archive maintenance performs the critical task of validating the health of the backup archive and checking for data corruption.Validation and healing

Item Description
1 Archive maintenance

The process of archive maintenance.

2 Validation If archive maintenance finds bad blocks of data, the destination self-heals the archive by re-requesting the affected data blocks from the source device.
3 Healing The source device sends new blocks of data to heal the archive.

Types of archive maintenance

The following table summarizes the differences between shallow and deep maintenance:

Prunes files Validates files and heals corruption Verifies block checksums Compacts archives Resource intensive Default interval
Shallow maintenance Yes Yes No No No 7 day
Deep maintenance Yes Yes Yes Yes Yes 28 days

Shallow maintenance

During shallow maintenance, the destination:

  • Checks for data corruption by making sure all files have valid versions and valid block lists
  • Prunes file versions and deleted files based on the device's frequency and version settings
  • Prunes deselected files and data excluded by file exclusions

Shallow maintenance runs on an archive every 7 days, or according to the configured schedule.

Deep maintenance

During deep maintenance, the destination:

  • Checks for data corruption by making sure all files have valid versions and valid block lists
  • Prunes file versions and deleted files based on the device's frequency and version settings
  • Prunes deselected files and data excluded by file exclusions
  • Validates block checksums for the entire archive
  • Compacts archives to reclaim disk space

Deep maintenance runs each time the last deep maintenance was more than 28 days ago, or if there is an error related to the archive. The deep maintenance interval is not configurable for cloud destinations, but can be for computer destinations.

Maintenance queue

All maintenance jobs get placed into a queue. This applies to shallow and deep maintenance jobs initiated by the user or the admin. You can view and cancel maintenance jobs under Settings > Server > Archive Maintenance Queue.

For cloud destinations, each server has two different maintenance queues - user and system:

  • User: Archive maintenance initiated by a user, either by clicking Compact in the CrashPlan app or from Device Details > Archive Action Menu in the administration console. All jobs in the user queue undergo deep maintenance.
  • System: Archive maintenance initiated by the system according to the configured maintenance interval, or in response to a detected error in the archive.

Both system and user maintenance queues display in the Archive Maintenance Queue.

Add a job to the archive maintenance queue

Follow these steps to manually add a job to your archive maintenance queue from the administration console:

  1. Open the administration console.
  2. Go to Devices.
  3. Select the CrashPlan device for which you want to start archive maintenance.
  4. From the device details, select the Archive Options icon Archive Options.
  5. Select Maintain Archive...
    Your job is added to the queue.

Customizing when maintenance occurs

The shallow maintenance interval is configurable on all destination types. In most environments, we recommend configuring the interval to seven days, which is the default. Archive maintenance is an I/O intensive operation, so a seven-day interval provides good distribution of work on the destination side, while also validating the health of the backup archives at a reasonable interval.

Cloud destinations

Each destination has its own archive maintenance interval that applies to all archives, store points, and storage servers within that destination. The deep maintenance interval is not configurable for cloud destinations. To configure shallow maintenance:

  1. Log in to the administration console.
  2. Go to Settings > Server.
  3. From the General section, change the setting for Maintain archives every to the desired value.

Computer and local folder destinations

Computer destinations are other devices that can be used as backup destinations. For example, a user with both a desktop and a laptop could use the desktop as a destination for the laptop, and vice versa.

Local folder destinations can be any folder or directory located on any disk that is physically mounted on a device running the CrashPlan app. However, local folder destinations are generally located on an external hard drive, USB drive, or other large capacity external storage device.

Maintenance scheduling
Local folder backups cannot be scheduled separately from other inbound computer backups. If a computer is backing up to a folder location and also receiving backups from another computer, maintenance for both backups will run on the same schedule.

For computer-to-computer backup, perform the steps below for the destination computer. For local folder backups, perform these steps for the source computer.

To configure the maintenance interval from the CrashPlan app:

  1. Open the CrashPlan app.
  2. Go to Settings > General > Inbound backup from other computers.
  3. Change the settings for Perform shallow maintenance and Perform deep maintenance.

To configure the maintenance interval from the administration console:

  1. Log in to the administration console.
  2. Navigate to the device you want to configure.
  3. Go to Action Menu > Edit > General.
    Ensure that Accept inbound backups is selected.
  4. Change the settings for Perform shallow maintenance and Perform deep maintenance.

How long does it take to see an increase in free space?

If a user or admin deletes files or folders, or adds new file exclusions, the free space available on the store point does not increase immediately. Pruning must occur during maintenance, and a compact must also occur. For example, if you exclude all mp3 files globally from existing backups, it may take up to a month before a compact operation removes all mp3 files from all backup archives. However, you will see increases in free space as backup archives undergo compaction during regular maintenance cycles. If you'd like the data removed sooner, you can trigger deep maintenance on individual archives from the device or from the console.

Performance considerations

Distribution of archive maintenance jobs

Each backup archive is on its own archive maintenance schedule. The destination distributes the workload so it does not process all maintenance jobs at once. For example, maintenance may begin on Wednesday at 4:09 pm on the archive associated with Joe's Macbook Pro, while maintenance for Michelle's Windows 8 desktop may begin Friday at 1:36 am.

Archive availability during maintenance

Backups and restores interrupt scheduled maintenance jobs. However, maintenance triggered by a user or admin, or triggered by the system due to an error, are not interrupted. Archives undergoing this type of maintenance are unavailable for restore until maintenance finishes. If maintenance is slow, check the Maintenance Rates:

  1. Go to Settings > Server > General.
  2. Click View next to Archive maintenance queue.
  3. From the action menu select Adjust Maintenance Rates.
  4. Move the slider controls to increase the priority given to user and system maintenance jobs, up to a maximum of 100 percent.

IOPS and disk I/O

  • CrashPlan PROe calls for fast disk I/O. You may see performance bottlenecks on network storage, such as a NAS or a SAN that is not fiber attached. Consider moving to storage devices with higher throughput and lower latency.
  • Triggered maintenance jobs and maintenance triggered by errors are always deep maintenance. Deep maintenance is more resource intensive than shallow maintenance.

Storage servers with multiple Store point

The enterprise server runs one of each maintenance job type (system and user) on each store point simultaneously. Storage servers with multiple store points may therefore run more than two maintenance jobs at a time. For example, a storage server with 4 store points could run up to 8 maintenance jobs simultaneously.

We recommend limiting the number of store points on each storage server, and taking into account the limitations of the hardware. Use destinations and organizations to assist in organizing users and archives, rather than using store points or storage servers.