Please help with my \"Move files to Folders\" script

This topic contains 5 replies, has 4 voices, and was last updated by Profile photo of random commandline random commandline 6 months, 3 weeks ago.

Viewing 6 posts - 1 through 6 (of 6 total)
  • Author
    Posts
  • #36046
    Profile photo of Scott Coats
    Scott Coats
    Participant

    I am a forum lurker and have learned SO much from PowerShell.org about scripting. I've been able to cobble together some really useful scripts to help in my daily life, but I have run in to a roadblock and can't seem to help myself out of it. I am NOT a programmer and really struggling to learn PowerShell to help me some routine but cumbersome tasks.

    I put a script together to help me move a large number of movie and pic files (only png, jpg, gif and mpg formats) to subfolders within the same directory.

    First Issue

    I have to have two versions of the script to limit the move to 5000 files where total files => 5000. But can't wrap my head around how to clean up the script to account for instances where there are <5000 files.

    The change I now make manually is from this:

    for ($i=0; $i -lt $files.Count; $i++) {
        $outfile = $files[$i].Name
    

    To this for when there are >5000 files:

    # Limit the move to the first 5000 items y changing "$files.count" to "5000".
    
    for ($i=0; $i -lt 5000; $i++) {
        $outfile = $files[$i].Name
    

    What I would like is one script that (1) counts the files and then (2) moves them appropriately with multiple folders of 5000 items where total files >= 5000.

    I want the ability to break up, say 22,000 files (of the same type) into five folders, To Sort 1, To Sort 2, etc. with To Sort 5 holding the last 2,000. What I am getting instead is just one folder without the numbering and subsequent folder creation (because I don't know how to do that) and the script craps out with an exception error where total files < 5000.Second IssueI would like to run the script from current directly without having to copy it there manually. In other words, one script that runs in multiple directories (Jan, Feb Mar, or whatever). I have no idea how to grab the current directory and have the script execute only on those files. I was using the recurse function when running the script on multiple subdirectories, but that's not helpful given the above limitations.If you could help (1) implement the folder creation and (2) suggest improvements, I would be greatly appreciative. I have really tried to solve this on my own, but I'm at a loss at this point.

    $files = Get-ChildItem -File -Recurse
    
    
    Here is my complete cobbled together script.
    
    
    # Move downloads to folders to make sorting easier.
    
    ################ SET THE SCRIPT PROCESSOR PRIORITY #####################
    
    (Get-Process -id $pid).PriorityClass = "High"
    
    
    ################# SORT THE FILES IN TO SUBDIRECTORIES ###################
    # Go through each file in the current working directory and run the script to move like files to a subdirectory.
    #
    # Uncomment "-Recurse" to run the routine on a batch of directories, not just the current one.
    
    $files = Get-ChildItem -File ##-Recurse
    
    # Limit the move to the first 5000 items y changing "$files.count" to "5000".
    
    for ($i=0; $i -lt $files.Count; $i++) {
        $outfile = $files[$i].Name
    
    # Move GIF's from the current directory to a new subdirectory called "GIF's to Sort"
        if ($outfile.EndsWith("gif")){
            mkdir -Path "GIFs to Sort" -Force
            move  $files[$i].FullName "GIFs to Sort" -Force
        }
    
    	# Move MP4's from the current directory to a subdirectory called "Movies to Sort"
        if ($outfile.EndsWith("mp4")){
            mkdir -Path "Movies to Sort" -Force
            move  $files[$i].FullName "Movies to Sort" -Force
        }
    
    # Move PNG's from the current directory to a subdirectory called "Pics to Sort"
    	if ($outfile.EndsWith("png")){
            mkdir -Path "Pics to Sort" -Force
            move  $files[$i].FullName "Pics to Sort" -Force
        }
    	
    # Move JPG's from the current directory to a subdirectory called "Pics to Sort"
    	    if ($outfile.EndsWith("jpg")){
            mkdir -Path "Pics to Sort" -Force
            move  $files[$i].FullName "Pics to Sort" -Force
        }
    }
    #36060
    Profile photo of Don Jones
    Don Jones
    Keymaster

    Try "Learn PowerShell in a Month of Lunches." It doesn't assume you're a programmer or try to make you into one.

    Frankly, I'd use Robocopy for this ;). I know it's less sexy somehow, but it's robust, reliable, and faster. Often much faster.

    Also, I'm not sure you're aware how impactful setting this to high priority is. If you are, cool. If you're not, well, just consider it.

    Anyway.

    Creating a new folder is the mkdir command.

    # this sets you up. this before your For loop.
    $currentSortFolder = 1
    $currentSortFolderName = "Sort $currentSortFolder"
    $filesInCurrentSortFolder = 0
    $maxFilesPerSortFolder = 5000
    
    # do this right inside your for loop
    $filesInCurrentSortFolder += 1
    
    if ($filesInCurrentSortFolder -gt $maxFilesPerSortFolder) {
      $currentSortFolder += 1
      $currentSortFolderName = "Sort $currentSortFolder"
    }
    
    if (-not (Test-Path $currentSortFolderName)) {
      mkdir $currentSortFolderName
    }
    

    I'm not saying that's exactly what you want to do – it's meant to lay out the logic I think you're asking for.

    As-is, your script is continually trying to run mkdir on folders that already exist, which is kind of ugly.

    #36071
    Profile photo of Scott Coats
    Scott Coats
    Participant

    Hi Don, thanks for the RoboCopy suggestion (and its cousin, RichCopy). I have used RoboCopy before for LAN and WAN connections, and it's a great alternative to FTP and much faster in my experience.

    These copies are being done locally, within the same RAID volume.

    The fundamental problem I am trying to solve is to limit the copy to 5000 files, then create the next folder for the next 5000 files.

    Secondarily, how to make the script location-aware so I can run it against the current folder.

    Regarding performance, the PowerShell environment is surprisingly fast when copying. It takes less than 20 seconds in my environment and I can't seem to find any facility within RoboCopy to limit the size of the folder/file combinations.

    Fundamentally, I think this is a scripting issue...but I am pretty clueless how to accomplish what I want to do within PowerShell.

    Thanks again. I appreciate the response.

    #36072
    Profile photo of Jeremy Murrah
    Jeremy Murrah
    Participant

    What sort of folder structure do you want to end up with? I think a lot of your folder confusion comes from assuming current directory in your commands. Consider using the -path and -destination parameters to specify exactly where to look, where to move to, etc.

    You can variable-ize all that and put the variable definitions at the top of your script, something like this:

    $SourceDirectory = 'c:\downloads'
    $DestinationRootDirectory = 'c:\PicstoSort'
    $GIFDestinationDirectory = "$destinationRootDirectory\GIF"
    $MP4DestinationDirectory = "$destinationRootDirectory\MP4"
    

    Or however you want to arrange it. This can also serve as documentation since you have that all laid out up front it makes it easy to see what you're doing and make changes as necessary. You could also put a comment block at the top to lay out the folder structure you want to end up with. Something like

    < #
    Folder structure after script is done should look like this
    C:\
    ---PicstoSort
    ------JPG
    ---------folder1 <---first batch of 5000
    ---------folder2 <---second batch of 5000
    ------GIF
    ---------folder1 <---first batch of 5000
    ---------folder2 <---second batch of 5000
    ------MP4
    ---------folder1 <---first batch of 5000
    ---------folder2 
    
    #36073
    Profile photo of Scott Coats
    Scott Coats
    Participant

    Hi Jeremy, you have the structure exactly correct. But how do I automate the folder creation/numbering?

    I can hard code the paths, I was primarily interested in the folder creation and limitation to 5000 items for each.

    Any thoughts?

    #36089
    Profile photo of random commandline
    random commandline
    Participant

    This script will ask you for source, destination, and how many per batch before moving files to subfolders. I asked for batch number in case you want to test this on a smaller scale before production. If you prefer to use Copy-Item, you can change that part of the switch.

    $source = Read-Host "Enter Source Path Here." 
    $dest = Read-Host "Enter Destination for sub folders Here."
    $batch = Read-Host "How many per batch?"
    $files = Get-ChildItem -Path $source -File -Recurse -Include '*.gif',
    '*.mp4','*.png','*.jpg'
    
    # Move every 5000 files to subfolders
    foreach ($a in '.gif','.mp4','.png','.jpg'){
    $count = $null ; $newcount = $batch ; $base = $a -replace '\.'
    switch ($files -match $a){
    $_ {$count++}
    $_ {Robocopy $_.Directory "$dest\$base\$newcount" $_.Name /MOV | Out-Null}
    $_ {If ($count % $batch -eq 0){$newcount = $count+$batch ; "Folder $count in $base Complete"}}
    }} # End switch,foreach
    

    This is the result.

    < #
    Folder structure after script is done should look like this
    ------JPG
    ---------5000 <---first batch of 5000
    ---------10000 <---second batch of 5000
    ------GIF
    ---------5000 <---second batch of 5000
    ---------10000 <---second batch of 5000
    ------MP4
    ---------5000 <---second batch of 5000
    ---------10000 <---second batch of 5000
Viewing 6 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.