Personal Project - HELP

This topic contains 3 replies, has 3 voices, and was last updated by  jdlech 3 weeks, 1 day ago.

  • Author
    Posts
  • #103229

    jdlech
    Participant

    First, let me give you the background. I have a 5TB HDD (Drive D:) I also have 4 smaller drives (M:,N:,O:, and P:).
    I also have just a couple of weeks coding experience (total noob). Which is why I've come here.
    What I want to do is a file by file copy from the source (D:) to the destination (M,N,O, or P) preserving the directory structure.
    The stumbling block is that many files and paths have spaces in them and possibly even escape characters.
    I originally wanted to build the code around the output of this line (attempting to pass it to a variable or fill an array).

    Get-ChildItem D:\ -Force -Recurse | Format-Table FullName

    My original idea was to split the output into 3 parts: the drive letter, the full path+file name (without the drive letter), and file length (which I know is not included in the line)
    But then I read that table formatted output is considered "bad form" and causes all manner of problems, so I abandoned it.
    So then, I managed to create this:

    cls
    Clear-RecycleBin -Force  # No point in backing up deleted files.
    $Source = "D:\"
    $target = 'M:'
    
        get-childitem -literalpath $Source -Force -recurse | Foreach-Object {
            $_.FulName, $_.Length
            # Get remaining space on 1st drive (M)
            $freeM = Get-PSDrive M |Select-Object -ExpandProperty Free
            if ($_.Length -lt  $freeM)
                 {
                 # Copy-Item
                 Copy-Item "$($_.fullName)" -Container -Verbose -Destination $target -Recurse -Force
                 }
            else {
                $target="N:"
                # Get space on N
                $freeN = Get-PSDrive N |Select-Object -ExpandProperty Free
                if ($_.Length -lt  $freeN) {
                    Copy-Item "$($_.fullName)" -Container -Verbose -Destination $target -Recurse -Force 
                }
                else {
                    $Target = "O:"
                    # Get space on O
                    $freeO = Get-PSDrive O |Select-Object -ExpandProperty Free
                    if ($_.Length -lt  $freeO) {
                        Copy-Item "$($_.fullName)" -Container -Verbose -Destination $target -Recurse -Force
                    }
                    else {
                        $Target = "P:"
                        # Get space on P
                        $freeP = Get-PSDrive P |Select-Object -ExpandProperty Free
                        if ($_.Length -lt  $freeP) { 
                            Copy-Item "$($_.fullName)" -Container -verbose -Destination $target -Recurse -Force
                        }
                        else {
                            Write-Host '"No space available, $($_.Directory)\\$($_.Name) not copied'
                        }
                    }
                }
            }
        }

    But this has a few problems. (the -verbose switch is only for troubleshooting purposes). The biggest problem is that it seems to be batch copying entire directories instead of file by file. So, when drive M became full, the script continued to try copying the contents of the directory; which returned a bunch of "drive full" errors. The errors ended when it moved on to the next directory. Suddenly, it started copying to drive N with the new directory, but a whole bunch of files in the last directory were never copied due to the "drive full" error.

    Can this be fixed, for should I abandon it and try another method?

    Also, there should be a check to see if the file already exists on any of the target drives. If so, skip to the next file. Or the opposite, if not, then copy it (if there's enough space).

    One last thing that's really bugging me. Notice I use $_.FulName (one L) in the Get-ChildItem line, but then use $_.FullName (two Ls) to copy. This works, but I don't know why? I tried to use 1 L everywhere and it broke the code. So if anyone can explain why that works, I'd be a much wiser man.

    Any help is greatly appreciated, of course.
    TIA.

  • #103235

    iain Barnetson
    Participant

    "file by file copy from the source ..... to the destination......preserving the directory structure"
    Robocopy is made for just that

  • #103282

    Nicholas J Crooks
    Participant

    Honestly I am not sure how you might accomplish this but it sounds to me like you may need something like a running size variable

    Please understand I am also a noob 🙂

    Hopefully you get better advice but I would suggest a couple of additional variables that will keep track of size

    $drives = array of drives

    $drive = current drive

    $runsize = $runsize + current directory to copy

    $targetSize = (gwmi or CIM of target drive for disk space) – $runSize

    "IF $runSize -lt $targetSize then robocopy /Mir /MT:X ....."
    else
    $drive changes/increments to next drive in $drives

    GOOD LUCK!

    • #103412

      jdlech
      Participant

      In fact, I've started an experimental code snippet that does exactly that. But in this case, I'm dumping the contents of all the drives into separate arrays. Then I'm going to compare the contents of each, so that I never overwrite any files. I'm experimenting with the split command (learning it, really), to strip the drive letter before comparing. I should end up copying anything that isn't already backed up.

      But right now, I'm running into a separate nagging problem.

      $item = Get-ChildItem -Path D: -Force -Recurse -Exclude *$Recycle*

      gets me this error

      Get-ChildItem : Access to the path 'D:\System Volume Information' is denied.
      At Z:\Backups\DataBackupScript\FileFinder.ps1:4 char:9
      + $item = Get-ChildItem -Path D: -Force -Recurse -Exclude *$Recycle*
      + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      + CategoryInfo : PermissionDenied: (D:\System Volume Information:String) [Get-ChildItem], UnauthorizedAccessE
      xception
      + FullyQualifiedErrorId : DirUnauthorizedAccessError,Microsoft.PowerShell.Commands.GetChildItemCommand

      And I can't figure out how to exclude the folder.

You must be logged in to reply to this topic.