Truncate Long File Extensions

This topic contains 11 replies, has 7 voices, and was last updated by Profile photo of Rimon Hasan Rimon Hasan 10 months, 2 weeks ago.

  • Author
    Posts
  • #30461
    Profile photo of Kawika Moss
    Kawika Moss
    Participant

    I have a function that searches a directory for any files that have extensions longer than 25 characters and will truncate it to 25 characters and then write the new path to a database. For the most part it works fine, but I have had some complaints that sometimes the function doesn't get all the files that have the long file extensions. However, if I go back and run it, it will fix it and do as it should. I need a review of my function to insure that I have not left a hole somewhere, which is allowing for these issues to happen.

    Function RunTruncateLongFileExtension
    {
    
        param
        (
    
            [Parameter(Position = 1, Mandatory = $true)] [string]$ServerInstance,
    		[Parameter(Position = 2, Mandatory = $true)] [string]$ProjDatabase,
            [Parameter(Position = 3, Mandatory = $true)] [string]$exportPath,
            [Parameter(Position = 4, Mandatory = $true)] [int]   $extLength,
            [Parameter(Position = 5, Mandatory = $true)] [string]$PsScriptPath
        )
        $startTime = Get-Date
    
        $fnName = $MyInvocation.MyCommand
        Write-Verbose -Message "Function Location: $fnName"
    
        # Load helper functions
    	. "$PsScriptPath\Functions\GetQueryOutput.ps1"
        
        # Identify all files with extensions longer than 25 characters
        $files = Get-ChildItem "$exportPath\NATIVE" -Recurse -File | Where-Object {$_.Extension.Length -ge $extLength}
    
        Write-Verbose -Message "Searching for long file extensions within the Legal Export..."
    
        ForEach ($file in $files)
        {
            $i++
    
            $filePath = "$exportPath\NATIVE"
    
            Write-Verbose -Message "Truncating File Extension: $file"
    
            # Get basename of original file to use for search after it is renamed
            $baseName =$file.BaseName
                    
            # Rename file extensions to x amount of characters after the (.)
            $file | Rename-Item -New "$($file.basename).$($file.extension.substring(1,$extLength))"
    
            #Search for renamed file in directories
            $renamedFileInfo = Get-ChildItem -Path "$filePath" -Filter "$baseName.*" -Recurse 
    
            # Get values required for database inserts
            $docId = $baseName
            $nativePath = $renamedFileInfo.FullName
    
            $getPathsQuery = "select 
                                    (select propertyValue from admin.properties where propertyName = 'nuixExportDirectory') as LocalPath,
                                    (select propertyValue from admin.properties where propertyName = 'nativeNetworkPath') as NetworkPath"
            
            $paths = (Get-QueryOutput $ServerInstance $ProjDatabase $getPathsQuery)        
            
            $localPath = $paths.LocalPath
            $networkPath = $paths.NetworkPath
            
            $nativePath = $nativePath -replace [System.Text.RegularExpressions.Regex]::Escape($localPath),$networkPath
                            
            $queryText = "IF NOT EXISTS (SELECT 1 FROM EXT.LegalExportDocumentMetric WHERE DocID = '$docId')
                            BEGIN; PRINT N'Failed'; END;
                            ELSE
                            BEGIN; UPDATE EXT.LegalExportDocumentMetric SET NativePath = '$nativePath' WHERE DocID = '$docId'; PRINT N'Success'; END;" 
                           
            $results = Get-QueryOutput $ServerInstance $ProjDatabase $QueryText -ErrorAction 'Stop'   
                
            If ($results -eq "Failed")
            {
                Write-Verbose -Message $Error[0] 
                return "Failed", $Error[0]         
            }
            Else
            {
                Write-Verbose -Message "An udpate has been applied to the NativePath field, for $DocID, in the EXT.LegalExportDocumentMetric table..."
            }  
    
        }  
        $endTime = Get-Date
        $duration = New-TimeSpan -Start $startTime -End $endTime
         
        $extLength = $extLength - 1
        If ($i -gt 0) 
        {        
            Write-Verbose -Message "There was $i file(s) found with file extensions longer than $extLength characters, the new Native Path has been recorded in the EXT.LegalExportDocumentMetric table...Duration: $duration"
    
            return "Success"
        }
        Else
        {
            Write-Verbose -Message "There were no files found with extensions longer that $extLength...Duration: $duration"
    
            return "No Extensions to truncate..."
        }    
    
    }
    
  • #30466
    Profile photo of Rob Simmers
    Rob Simmers
    Participant

    I think there is definitely some room for improvement in your logic and how you are detecting errors. Rather than running a second Get-ChildItem, take a look at this approach and see if make sense to you:

    [CmdletBinding()]
    param ()
    
    Function Rename-LongFileExtension {
        [CmdletBinding()]
        param (
            [Parameter(Position = 3, Mandatory = $true)] [string]$exportPath,
            [Parameter(Position = 4, Mandatory = $true)] [int]$extLength
        )
        begin { }
        process {
            Write-Verbose -Message ("Function Location: {0}" -f $MyInvocation.MyCommand)
            
            # Identify all files with extensions longer than 25 characters
            # ($extLength + 1) to account for $_.Extension contains a dot (.)
            $files = Get-ChildItem "$exportPath\NATIVE" -Recurse -File | Where-Object { $_.Extension.Length -gt ($extLength +1) }
            
            Write-Verbose -Message "Searching for long file extensions within the Legal Export..."
            Write-Verbose -Message ("There are {0} file(s) found with file extensions longer than {1} characters" -f $files.Count, $extLength)
            #Process only if the Get-ChildItem query returned something
            if ($files) {
                $results = ForEach ($file in $files) {
                    # Get basename of original file to use for search after it is renamed
                    $currentFileName = $file.Name
                    $baseName = $file.BaseName
                    $newExt = $file.extension.substring(1, $extLength)
                    $newFileName = "{0}.{1}" -f $baseName, $newExt
                    
                    Write-Verbose -Message ("Renaming {0} to {1}" -f $currentFileName, $newFileName)
                    # Rename file extensions to x amount of characters after the (.)
                    try {
                        $file | Rename-Item -New $newFileName -Force -ErrorAction Stop
                        $status = "Success"
                    }
                    catch {
                        $status = "Failed: {0}" -f $_.Exception.Message
                        Write-Verbose -Message $status
                    }
                    
                    New-Object -TypeName PSObject -Property @{
                        FileName = $currentFileName;
                        NewFileName = $newFileName;
                        Status = $status;
                    } #New-Object
                } #foreach file
            } #if files not null
            else {
                Write-Verbose -Message "No files found to truncate"
            }
        } #process
        end {
            $results
        }#end
    } #Rename-LongFileExtension
    
    
    $startTime = Get-Date
    $results = Rename-LongFileExtension -exportPath "C:\Temp" -extLength 2 -Verbose
    $results | Format-Table -AutoSize
    #The results of the file operation are now in a PSObject, then
    #create a function to send them to the database
    #if($results) {$results | Set-MyDatabaseFunction}
    $endTime = Get-Date
    $duration = New-TimeSpan -Start $startTime -End $endTime
    Write-Verbose -Message ("Took {0:g} to complete" -f $duration)
    
    

    Output:

    VERBOSE: Function Location: Rename-LongFileExtension
    VERBOSE: Searching for long file extensions within the Legal Export...
    VERBOSE: There are 4 file(s) found with file extensions longer than 2 characters
    VERBOSE: Renaming test.csv to test.cs
    VERBOSE: Renaming test.ini to test.in
    VERBOSE: Renaming test.txt to test.tx
    VERBOSE: Renaming test.xml to test.xm
    
    Status  NewFileName FileName
    ------  ----------- --------
    Success test.cs     test.csv
    Success test.in     test.ini
    Success test.tx     test.txt
    Success test.xm     test.xml
    
    VERBOSE: Took 0:00:00.1093746 to complete
    

    Edit: Added -ErrorAction Stop after Rename-Item

  • #30469
    Profile photo of Kawika Moss
    Kawika Moss
    Participant

    Thank You! I will test with this...

  • #30471
    Profile photo of Kawika Moss
    Kawika Moss
    Participant

    This is awesome actually, not only is it more efficient than my original, but it helped be figure out why I was getting this issue. The files that were missed, had super long names and the limitation was reach with Get-ChildItem:

    Get-ChildItem : The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
    At \\...\RunTruncateLongFileExtension.ps1:53 char:18
    +         $files = Get-ChildItem "$exportPath\NATIVE" -Recurse -File | Where-Objec ...
    +                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : ReadError: (D:\Case\0012_H1...12711-0012-\020:String) [Get-ChildItem], PathTooLongException
        + FullyQualifiedErrorId : DirIOError,Microsoft.PowerShell.Commands.GetChildItemCommand
    

    Do you know a good work around for this? I've seen a few different options online, but wanted to check here first...

    Thanks,

  • #30491
    Profile photo of Brian B
    Brian B
    Participant

    How long of a path is your ExportPath ? Is it something where you could temporarily either map a drive or create a new PSDrive much further into the file system so D:\Case\kljlkjadfa\lkjlkjdaf\kjakljfa could be treated as Y:\ and then the remaining path would be that much shorter? Or do you actually not have much you can truncate?

    I would see it be something like this:

    Create a function to do the GCI on the folder you specify (already doing this) , trap any errors, create a new PSDrive further up the path that gave you a problem, GCI again remembering what the actual path was if you're reporting that.... and iterate though all folders that way.

    It would be horribly inefficient and i'm not sure on the exact code to do it cleanly, but that's how i'm seeing this getting accomplished.

    If your ExportPath variable is already a fairly lengthy path, simply creating a new PS Drive at the beginning of the script and using THAT as your ExportPath may be enough, though it doesn't cover circumstances where you still run into issues beyond the length you were able to shorten up.

  • #30566
    Profile photo of Kawika Moss
    Kawika Moss
    Participant

    What about using the Windows API to do it, are you or anyone familiar with that?

  • #30603
    Profile photo of David Schmidtberger
    David Schmidtberger
    Participant

    I've been using alpha to deal with the length limit.

    first import the DLL as a module, and then it is a simple as: something like this:
    [alphaleonis.win32.filesystem.directory]::Enumeratefiles($inputpath,'*',[System.IO.SearchOption]::AllDirectories)

    documentation on the enumerate directories call is here:
    http://alphafs.alphaleonis.com/doc/2.0/api/html/D91BDFAC.htm

    took a lot of playing around to figure out the calls, but this allowed me to get around the length limit

  • #30604
    Profile photo of Kawika Moss
    Kawika Moss
    Participant

    With this solution, are you able to do a Rename-Item as well, so that you can truncate the file name, making it usable?

    And, do you mind sharing an example of how you applied this in a PowerShell script?

  • #30607
    Profile photo of David Schmidtberger
    David Schmidtberger
    Participant

    here is a portion of the script I wrote that had this functionality in it:

    $folders = [alphaleonis.win32.filesystem.directory]::EnumerateDirectories($inputpath,'*',[System.IO.SearchOption]::AllDirectories)
    	foreach ($dir in $folders)
    		{
    		Write-Host $($dir)
    		try
    			{
    			
    			$directory = [alphaleonis.win32.filesystem.directory]::getaccesscontrol($dir) | select-object * -expandproperty access | select filesystemrights,identityreference,inheritanceFlags 
    			foreach ($entry in $directory)
    				{
    				try
    					{
    					$Properties = 
    					@{
    					Path = $dir
    					Permission = $entry.filesystemrights
    					Identity = $entry.identityreference
    					Inheritance = $entry.inheritanceflags
    					}
    					$Results += New-Object psobject -Property $Properties
    					}
    				catch
    					{
    					Write-Host "error processing $entry"
    					}
    				}
    			}
    		catch
    			{
    			write-host "error enumerating $($inputpath)"
    			}
    		}
    	$Results | Select-Object Path,Permission,Identity,Inheritance| Export-Csv $output -NoTypeInformation
    	$Results = @()
    

    first look I didn't see an obvious method for a simple rename. you might have to do something like write the file with the new name, and then delete the old one.

    the full documentation for alpha is located here (someday I hope someone will write a module to replace all of the *-item cmdlets using alphafs to make everyones lives easier)

  • #30608
    Profile photo of Mark Hammonds
    Mark Hammonds
    Participant

    here is another option I found worked really well. Boe Prox uses Robocopy in List only mode to populate an array of all files names from that you are able to use the Count property to find the ones that have a long path with that I was able to move the files. this has worked 100% for me with out having to load any special module.. Awesome workaround since Move-Item / Get-ChildItem suffers from the same limitation as network paths

  • #31553
    Profile photo of oxishmit
    oxishmit
    Participant

    Long Path Tool is awesome solution too

  • #36115
    Profile photo of Rimon Hasan
    Rimon Hasan
    Participant

    I am agree with oxishmit. Long Path Tool solved my problem.

    Oxana

You must be logged in to reply to this topic.