Array Purger

This topic contains 0 replies, has 1 voice, and was last updated by Profile photo of Forums Archives Forums Archives 5 years, 5 months ago.

  • Author
    Posts
  • #5100

    by TheFridgeMaster at 2012-09-11 19:33:51

    Good Morning All,

    I have very amateur coding skill (can handle myself in batch and vbs with the help of google)

    I am currently attempting to create a system (powershell script) to delete files (.mp3) recursively throughout user folders.

    gc mp3nodupes.txt |% {gci "c:\admin\users" -include $_ -recurse} |% {remove-item $_ -whatif}

    I want the script to read "mp3nodupes.txt" (which just contains a list of songs, no filepath, just filenames) for the filenames and then check the directories recursively for the names then delete them if it finds anything. but also keeping any mp3's that aren't in the list, intact.

    Are you following me? I feel like I have worded this in the worst possible way ha ha.

    I got the above code snippet from somebody trying to achieve something similar but I believe his list file contained the file paths as well.

    I have broken down the script to diagnose where it goes wrong and it appears the deletion part of the script is not working. it pipes the list in fine and searches recursively and returns back all the mp3's it finds but it does not delete them.

    I'm sorry if this is confusing, I look forward to your response 😀

    Thanks,
    Kyle.

    by DonJ at 2012-09-11 19:45:12

    I'm not sure you need the final ForEach (%). You should be able to just pipe to remove-object. But that shouldn't be your problem.

    You do know that -whatif makes the command not do anything, right? You've tested without that?

    by TheFridgeMaster at 2012-09-11 19:56:03

    [quote="DonJ"]I'm not sure you need the final ForEach (%). You should be able to just pipe to remove-object. But that shouldn't be your problem.

    You do know that -whatif makes the command not do anything, right? You've tested without that?[/quote]

    I did not know that. But I removed the -whatif and exected but still no deletion.

    by DonJ at 2012-09-11 20:01:17

    So, just remove the third pipeline segment – the remove bit. Does it produce output?

    If it produces output, and its the files you want to delete, try just manually deleting one using the remove-item command. See what happens.

    If remove-item is getting input, it'll delete the file unless you don't have permission to do so. So either you don't have permission, or it isn't getting any input from the prior command.

    by TheFridgeMaster at 2012-09-11 20:35:54

    [quote="DonJ"]So, just remove the third pipeline segment – the remove bit. Does it produce output?

    If it produces output, and its the files you want to delete, try just manually deleting one using the remove-item command. See what happens.

    If remove-item is getting input, it'll delete the file unless you don't have permission to do so. So either you don't have permission, or it isn't getting any input from the prior command.[/quote]

    Returns the same error and doesn't list any files.

    Get-ChildItem : The specified wildcard pattern is not valid: I Can Only Imagine (feat. Chris Brown & Lil Wayne) [Lyric....mp3
    At C:\listdelete.ps1:1 char:26
    + gc mp3nodupes.txt |% {gci < <<< "c:\admin\users" -include $_ -recurse}
    + CategoryInfo : NotSpecified: (:) [Get-ChildItem], WildcardPatternException
    + FullyQualifiedErrorId : RuntimeException,Microsoft.PowerShell.Commands.GetChildItemCommand

    by coderaven at 2012-09-11 21:23:18

    Here is what I suggest.

    Get-ChildItem -Path C:\admin\users -Recurse | foreach-object { foreach ($Song in (Get-Content mp3nodupes.txt)) {if ($_.Name -eq $Song) { Remove-Item -Path $_.FullName -WhatIf}}}

    I left the -Whatif on there so you have to modify it to actually remove an item.

    by TheFridgeMaster at 2012-09-11 21:56:15

    [quote="coderaven"]Here is what I suggest.

    Get-ChildItem -Path C:\admin\users -Recurse | foreach-object { foreach ($Song in (Get-Content mp3nodupes.txt)) {if ($_.Name -eq $Song) { Remove-Item -Path $_.FullName -WhatIf}}}

    I left the -Whatif on there so you have to modify it to actually remove an item.[/quote]

    I tried this and it did not return any errors this time but it does not delete any files. is there anyway to get a more verbose output so we can see where it is dropping the ball?

    Cheers.

    by TheFridgeMaster at 2012-09-11 22:14:26

    I tried running with Get-ChildItem -Path C:\admin\users -Recurse | foreach-object { foreach ($Song in (Get-Content C:\mp3nodupes.txt)) {if ($_.Name -eq $Song) { console.write -Path $_.FullName }}}

    and tried

    Get-ChildItem -Path C:\admin\users -Recurse | foreach-object { foreach ($Song in (Get-Content C:\mp3nodupes.txt)) {if ($_.Name -eq $Song) { write-output -Path $_.FullName }}}

    but it didn't output anything back. the script simply runs and then ends after a short while.

    by poshoholic at 2012-09-12 04:56:17

    How about this:

    $filesToKeep = Get-Content C:\mp3nodupes.txt
    Get-ChildItem -LiteralPath C:\admin\users -Recurse -Force -Filter *.mp3 | Where-Object {$filesToKeep -notcontains $_.Name} | Remove-Item -WhatIf

    This assumes that filesToKeep will contain filenames with no paths.

    A few points about how this is written that may be helpful:
    1. Using -Filter results in much faster Get-ChildItem processing, especially when you are doing recursive search.
    2. Using -Force in Get-ChildItem finds hidden files as well, so it's good to use it if you want to be thorough.
    3. Reading your content from the file once and storing it at the beginning is much faster than reading it for each file you get back from Get-ChildItem.
    4. The -contains and -notcontains operators are great for including/excluding items based on array membership. Much simpler than using foreach.

    Of course using -WhatIf here is just a safeguard to see what this might do on your system. Once you get the it way you like, you can remove the -WhatIf switch.

    by TheFridgeMaster at 2012-09-12 16:49:11

    [quote="poshoholic"]How about this:

    $filesToKeep = Get-Content C:\mp3nodupes.txt
    Get-ChildItem -LiteralPath C:\admin\users -Recurse -Force -Filter *.mp3 | Where-Object {$filesToKeep -notcontains $_.Name} | Remove-Item -WhatIf

    This assumes that filesToKeep will contain filenames with no paths.

    A few points about how this is written that may be helpful:
    1. Using -Filter results in much faster Get-ChildItem processing, especially when you are doing recursive search.
    2. Using -Force in Get-ChildItem finds hidden files as well, so it's good to use it if you want to be thorough.
    3. Reading your content from the file once and storing it at the beginning is much faster than reading it for each file you get back from Get-ChildItem.
    4. The -contains and -notcontains operators are great for including/excluding items based on array membership. Much simpler than using foreach.

    Of course using -WhatIf here is just a safeguard to see what this might do on your system. Once you get the it way you like, you can remove the -WhatIf switch.[/quote]

    Thanks for this, it is listing and deleting mp3 files but it is deleting ALL mp3 files. not just the ones listed in the mp3nodupes.

    switching -notcontain to -contain lists nothing and deletes no files.

    by TheFridgeMaster at 2012-09-12 18:22:56

    Upon further reading and research I have discovered the '-like' operator and it returns interesting results.

    $filesToRemove = Get-Content C:\mp3nodupes.txt
    Get-ChildItem -LiteralPath C:\admin\users -Recurse -Force -Filter *.mp3 | Where-Object {$_.Name -like $filesToRemove} | Remove-Item -whatif

    It returns an error for every child item it finds attached with "the sepcified wildcard pattern is not valid:" then a large string of every single song in the list. so I have got it to accept the array but it is pulling the whole thing as an individual filename.

    by TheFridgeMaster at 2012-09-12 20:29:03

    If I change the where to foreach I get true/false output from console.

    $filesToKeep = Get-Content C:\mp3nodupes.txt
    Get-ChildItem -LiteralPath C:\admin\users -Recurse -Force -Filter *.mp3 | Foreach-Object {$filesToKeep -notcontains $_.fullName} | Remove-Item -WhatIf

    Returns:
    Remove-Item : Cannot find path 'C:\True' because it does not exist.
    At C:\listdelete.ps1:2 char:132
    + ... $_.fullName} | Remove-Item -WhatIf
    + ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo : ObjectNotFound: (C:\True:String) [Remove-Item], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand]

    by poshoholic at 2012-09-13 05:23:00

    The first results you shared from my last reply identify that $filesToKeep does not contain any of the filenames that are found. If you know this is not the case, then we have an issue with the contents of the C:\mp3nodupes.txt file. Can you double-check the contents of your C:\mp3nodupes.txt file? Does it contain one mp3 file listed on each line in the file, with no leading or trailing spaces or commas or anything like that? That's the expected format to get the content into an array. Also, does it contain the filename only and not a path?

    i.e. The last script I shared is expecting it to contain something like this:

    Filename1.mp3
    Filename2.mp3
    Filename3.mp3

    If it is not in that format, then share the format and we can modify how we read in the mp3 file listing. For example, if it instead contains a list like this: Filename1.mp3,Filename2.mp3,Filename3.mp3, then we would need to read that content like this:
    $filesToRemove = (Get-Content C:\mp3nodupes.txt) -split ','
    Focus on just that one line of script, and then output the results of $filesToRemove like this to make sure it contains what you want.
    $filesToRemove | ForEach-Object {"'$_'"}
    By wrapping the file names in single quotes, that snippet should identify whether there are extra spaces surrounding the file names or not (we want to make sure we have just the file names, with no leading or trailing spaces, so that we can get an exact match in the Where-Object clause).

    by TheFridgeMaster at 2012-09-16 21:43:31

    [quote="poshoholic"]The first results you shared from my last reply identify that $filesToKeep does not contain any of the filenames that are found. If you know this is not the case, then we have an issue with the contents of the C:\mp3nodupes.txt file. Can you double-check the contents of your C:\mp3nodupes.txt file? Does it contain one mp3 file listed on each line in the file, with no leading or trailing spaces or commas or anything like that? That's the expected format to get the content into an array. Also, does it contain the filename only and not a path?[/quote]

    It did indeed have trailing spaces. /facepalm.

    Apologies for the late response.

    I send this script to a friend of mine who is a web developer and he decided to be really amazing and learn the syntax of powershell so he could write me a script. it works really well and I thought you might all be interested.

    #dofridgesjob.ps1
    #usage: dofridgesjob.ps1
    #follow prompts

    #clear the screen! I hate a busy console
    Clear-Host

    #set up user confirmation - yes = carebare mode, no = HARDCORE FUCK SHIT UP MODE
    $confirm = Read-Host "Do you want to run this script in safe-mode? Type YES or NO to continue"

    #user types an answer. failure to type yes or no will exit the script
    if ($confirm -eq "NO") {
    $flag = 1
    "Brave man. Let's hope this works!"
    $lognote = "Purging files: "
    }
    elseif ($confirm -eq "yes"){
    $flag = 0
    "Safe mode enabled."
    $lognote = "Pretending to purge these files: "
    }
    else{
    "Due to user being unable to follow simple instructions, this script will now exit."
    exit
    }

    #set up the array to store the located files
    $found_files = @()
    $space = 0
    #change this bit to your search location
    Get-ChildItem -Path C:\admin\users -Recurse -Force -Filter *.mp3 |
    % {
    #this part checks to see if there is a match.
    #if so, pop it on the array, otherwise skip it
    #also - get content will need to be changed to your dirty list
    foreach ($name in (Get-Content C:\mp3nodupes.txt)) {
    if ($_.Name -eq $name) {
    $found_files += $_.FullName
    $currentFile = $_.Fullname
    $currentFileSize = Get-Item $CurrentFile #not strictly the filesize
    $space += $currentFileSize.length #but we operate on it here as filesize
    }
    }
    }

    #for verbose output at the end
    $total_files = $found_files.length

    #delete or pretend to delete.
    foreach ($file in $found_files){
    if ($flag -eq 1){
    Remove-Item $file
    }
    else{
    Remove-Item $file -WhatIf
    }
    }

    #setting up to do put the space saved in human readable format
    #set it to bytes as a default
    $saved_space = [Math]::Round($space, 3)
    $saved_space_suffix = "bytes"
    if ($space -ge 1024){
    #only perform the next tier action if we NEED to
    $spaceKB = $space/1KB
    $saved_space = [Math]::Round($spaceKB, 3)
    $saved_space_suffix = "KB"
    }

    #only do these operations if we need to - as $spaceKB & MB will be set to 0 by default
    if ($spaceKB -ge 1024){
    $spaceMB = $space/1MB # < ----
    $saved_space = [Math]::Round($spaceMB, 3)
    $saved_space_suffix = "MB"
    }
    if ($spaceMB -ge 1024){
    $spaceGB = $space/1GB # < ----
    $saved_space = [Math]::Round($spaceGB, 3)
    $saved_space_suffix = "GB"
    }

    #logging - creates a text file with the date and time
    # - at the time of execution - and writes the array of matched files we have.
    $now = Get-Date -format "dd-MMM-yyy_hh-mm-ss"
    New-Item -path c:\scriptslog -name "$now.txt" -type file
    Add-Content -Path c:\scriptslog\$now.txt -value "Found $total_files items for purging, saving $saved_space $saved_space_suffix"
    Add-Content -Path c:\scriptslog\$now.txt -value $lognote
    Add-Content -Path c:\scriptslog\$now.txt -value $found_files

    ""#this is my version of \r\n
    ""#ghetto carriage return & newlines
    "Script cleaned $total_files files, saving $saved_space $saved_space_suffix"
    ""
    "Log output located at c:\scriptslog\$now.txt"
    ""
    "Script complete."

    enjoy!

    "edit: added space saving calculations"

You must be logged in to reply to this topic.