Script to clean backup files

Welcome Forums General PowerShell Q&A Script to clean backup files

Viewing 17 reply threads
  • Author
    Posts
    • #221475
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      I am working on creating a small script that will help me clean up a backup folder. Currently the backups are done nightly at 10 pm.

      I would like to trim files somewhat like this;
      -Keep each day for 4 weeks
      -Week 5 to lets say 26 keep only first day of week backups (Mondays)
      Week 27 to week 52 keep only first day of the month. (i might not keep this last clean as weekly might leave me enough room).

      One small issue we have is some days the backup is doubled. This is an issue from the vendor and cannot be fixed at this time, so I would like to delete backups that have more than 1 backup in day. I guess this can be performed before the other checks and trimming.

      I was wondering if there is anything similar already out there that can give me a starting point or if anyone can help me with small bits of this?

    • #221481
      Participant
      Topics: 4
      Replies: 2247
      Points: 5,484
      Helping Hand
      Rank: Community MVP

      Simon, welcome to Powershell.org. Please take a moment and read the very first post on top of the list of this forum: Read Me Before Posting! You’ll be Glad You Did!.

      This forum is for scripting questions rather than script requests. We do not write customized and ready to use scripts or solutions on request.

      Did you try to search for it. Usually there are already scripts available you could adapt to your special needs. You could search PowershellGallery or StackOverflow or simply use your favorite internet search engine.

    • #221499
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      I did read the rules and not asking to write a script. I was asking if anyone knew of something similar and I have used my favorite search engine and cannot seem to find anything.

      My first attack is to get a file list using a filter, but the filter doesnt seem to be working, The bellow code doesnt work when I am including the Where-Object, almost like its being ignored. When i just use the Get-ChildItem $path, it lists all my files.

      $path = "o:\test"

      Get-ChildItem $path -recurse |
      Where-Object { $_.CreationTime -ge "03/01/2020" -and $_.CreationTime -le "04/22/2020" }

       

    • #221514
      Participant
      Topics: 3
      Replies: 340
      Points: 1,120
      Helping Hand
      Rank: Community Hero

      Your code worked fine for me. You’re certain there are files in o:\test?

    • #221520
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      Yes when i remove the line

      Where-Object { $_.CreationTime -ge "03/01/2020" -and $_.CreationTime -le "04/22/2020" }

      it displays all my files in that test folder. The files i have are from 4/13/2020 to 4/21/2020

    • #221538
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      Your code worked fine for me. You’re certain there are files in o:\test?

      ok when i copied the files for test filtering it changed the createdtime for all of them to today’s date, only noticed after forcing powershell to display the creationtime field.

      Guess part one of my solution is completed.

    • #221565
      Participant
      Topics: 4
      Replies: 2247
      Points: 5,484
      Helping Hand
      Rank: Community MVP

      … and I have used my favorite search engine and cannot seem to find anything.

      Google -> powershell delete files older than

    • #221595
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      … and I have used my favorite search engine and cannot seem to find anything.

      Google -> powershell delete files older than

      I have been using scripts from this search for years, but my original post is a more advanced than just a simple remove old files.

    • #221670
      Participant
      Topics: 4
      Replies: 2247
      Points: 5,484
      Helping Hand
      Rank: Community MVP

      OK, try this as a starter …

      $root = 'D:\someBackupRootFolder'
      Get-ChildItem -Path $root -Recurse -File |
          ForEach-Object {
              $fileAge = New-TimeSpan -Start ($_.LastWriteTime).Date -End (Get-Date).Date
              if(
                  $fileAge.Days -gt 28 -and
                  $fileAge.Days -lt 182 -and
                  $_.LastWriteTime.DayOfWeek -ne 'Monday'
              ){
                  $_
              }
          }

      That covers your first 2 conditions. Try to add the last one yourself, ok? πŸ˜‰

    • #222243
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      After allot of testing i achieved pretty much what i was looking for. This will also be handy for other projects for backup manipulation. Thank you all for the help and bits of code.

      $path = 'c:\Backup'
      $archivePath = 'c:\Backup Archive'
      $fileExtension   = '*.zip'
      
      $weeklyStartDay = 29
      $weeklyEndDay = 182
      $archiveDays = $weeklyEndDay + 1        # Set days for archiving backups to different folder
      
      # delete duplicate files for daily backups
      echo 'Clean duplicates'
      Get-ChildItem -Path $path -Filter $fileExtension -File |                # get files that start with the prefix and have the extension '.zip'
          Where-Object { $_.BaseName -match '_\d{14}$' } |                    # that end with an underscore followed by 14 digits
          Group-Object -Property @{Expression = { $_.LastWriteTime.Date }} |  # create groups based on the date part without time part
              ForEach-Object {
                  $_.Group | 
                  Sort-Object -Property LastWriteTime -Descending |           # sort on the LastWriteTime property
                  Select-Object -Skip 1 |                                     # select them all except for the first (most recent) one
                  Remove-Item -Force                                   # delete these files
              }
      
      # after X days, remove all backups except Mondays
      echo 'Clean older bacckup files, keep Mondays'
      Get-ChildItem -Path $path -File |
          ForEach-Object {
              $fileAge = New-TimeSpan -Start $_.LastWriteTime -End (Get-Date).Date
              if(
                  $fileAge.Days -ge $weeklyStartDay -and
                  #$fileAge.Days -le $weeklyEndDay -and
                  $_.LastWriteTime.DayOfWeek -ne 'Monday'
              ){
                  #$_
                  Remove-Item -Path $_.FullName -Force 
              }
          }
      
      # archive files older then specified date
      echo "Archive older backups"
      If(!(test-path $archivePath)) { New-Item -ItemType Directory -Force -Path $archivePath | Out-Null }
      Get-ChildItem -Path $path -File |
          ForEach-Object {
              $fileAge = New-TimeSpan -Start $_.LastWriteTime -End (Get-Date).Date
              if(
                  $fileAge.Days -ge $archiveDays
              ){
                  #$_
                  Move-Item -Path $_.FullName -Destination $archivePath 
              }
          }
    • #222267
      Participant
      Topics: 4
      Replies: 2247
      Points: 5,484
      Helping Hand
      Rank: Community MVP

      Simon,
      Great. Thanks for sharing.

      May I suggest some little tweaks anyway? πŸ˜‰

      You use Get-ChildItem for almost the same files 3 times. That’s going to take a lot of time, which you can avoid. Simply create the list of files to process once and use it as often as you need.
      Something like this …

      $path = 'c:\Backup'
      $archivePath = 'c:\Backup Archive'
      $fileExtension = '*.zip'
      
      $weeklyStartDay = 29
      $weeklyEndDay = 182
      $archiveDays = $weeklyEndDay - 1
      
      $CompleteBakupFileList = Get-ChildItem -Path $path -Filter $fileExtension -File |
                              Select-Object -Property *, @{Name = 'fileAgeDays'; Expression = { (New-TimeSpan -Start $_.LastWriteTime -End (Get-Date).Date).Days } },
                                                          @{Name = 'DayOfWeek'; Expression = { $_.LastWriteTime.DayOfWeek } }
      If (-not (test-path $archivePath)) { New-Item -ItemType Directory -Force -Path $archivePath | Out-Null }
      
      $CompleteBakupFileList |
      Where-Object { $_.BaseName -match '_\d{14}$' } |
      Group-Object -Property @{Expression = { $_.LastWriteTime.Date } } |
      ForEach-Object {
          $_.Group | 
          Sort-Object -Property LastWriteTime -Descending |
          Select-Object -Skip 1 |
          Remove-Item -Force
      }
      foreach ($BackupFile in $CompleteBakupFileList) {
          if (
              $BackupFile.fileAgeDays -le $archiveDays
          ) {
              Move-Item -Path  $BackupFile.FullName -Destination $archivePath 
          }
          elseif (
              $BackupFile.fileAgeDays -le $weeklyStartDay -and
              $BackupFile.fileAgeDays -ge $weeklyEndDay -and
              $BackupFile.DayOfWeek -ne 'Monday'
          ) {
              Remove-Item -Path $BackupFile.FullName -Force
          }
      }
    • #222420
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      Olaf I received an email you had suggested a few modification of the code but cant see it here, was it removed?

    • #222435
      Participant
      Topics: 4
      Replies: 2247
      Points: 5,484
      Helping Hand
      Rank: Community MVP

      The SPAM detection of the forum is quite picky. It got on hold for moderation. I already asked to release it. It might taky some time. πŸ˜‰

    • #223149
      Senior Moderator
      Topics: 8
      Replies: 1215
      Points: 4,335
      Helping Hand
      Rank: Community Hero

      Its released from spam now πŸ™‚

    • #223167
      Participant
      Topics: 4
      Replies: 2247
      Points: 5,484
      Helping Hand
      Rank: Community MVP

      Great. Thanks. πŸ˜€

    • #224193
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      Small bug with your suggestion Olaf, line 30 and 31 should be reversed for the logic of -ge and -le

      $BackupFile.fileAgeDays -le $weeklyStartDay -and
      $BackupFile.fileAgeDays -ge $weeklyEndDay -and

      should be

       $BackupFile.fileAgeDays -ge $weeklyStartDay -and
      $BackupFile.fileAgeDays -le $weeklyEndDay -and
    • #224229
      Participant
      Topics: 4
      Replies: 2247
      Points: 5,484
      Helping Hand
      Rank: Community MVP

      Small bug with your suggestion Olaf, line 30 and 31 should be reversed for the logic of -ge and -le

      I hope you catched that before you rolled it out to production. πŸ˜‰ But as you’re not complaining about more than that the rest seems to work as expected!? πŸ˜‰

    • #224232
      Participant
      Topics: 1
      Replies: 8
      Points: 33
      Rank: Member

      hahaha thats where the -WhatIf comes in handy. Yes everything else works perfectly thank you again for the help.

Viewing 17 reply threads
  • You must be logged in to reply to this topic.