Help needed doing a find/replace on a group of text files

This topic contains 9 replies, has 4 voices, and was last updated by Profile photo of Adam Bertram Adam Bertram 2 years, 5 months ago.

  • Author
    Posts
  • #16797
    Profile photo of Adam Bertram
    Adam Bertram
    Participant

    I've been working on this insane issue for at least 2 hours now and I've all but had it. I've got a list of text files I'd like to do a find/replace on for a specific string. I've tried all kinds of combinations of using Set-Content on the pipeline, Set-Content by itself, Out-File on the pipeline and even deleting and creating a new file via Add-Content and nothing works. I fear at this point my frustration is so high due to such a simple request I'm going in circles.

    The text file that I'm testing with is D:\MDTBuildLab\Control\CustomSettings.ini. It has a line in it with the string "WSUSServer=http://mdt-01:8530. I want to replace the string MDT-01 and mdt-01 with the string replacethis. I've confirmed that it's replacing MDT-01 but not mdt-01. Also, I'm running this from a WinForm. I originally thought it might be due to that but it IS replacing MDT-01 just fine. Why would it discriminate?

    I'm not sure why such a simple task doesn't work. I've narrowed it down to the writing of the file, I believe. If I echo out the $NewContent variable, it shows my replaced text but as soon as I look at the file after it's done, it is not changed. I get no errors or warnings when doing this.

    $TextFiles = @('D:\MDTProduction\Control\Bootstrap.ini',
    			'D:\MDTProduction\Control\Customsettings.ini',
    			'D:\MDTBuildLab\Control\Bootstrap.ini',
    			'D:\MDTBuildLab\Control\Customsettings.ini',
    			'D:\MDTBuildLab\Scripts\Realocate.cmd'
    		)
    foreach ($FilePath in $TextFiles) {
      Write-Verbose "Searching for string in $FilePath..."
      $Content = Get-Content $FilePath
      if ($Content -match 'mdt-01') {
        Write-Verbose "Found mdt-01 match in $FilePath"
        $NewContent = $Content | % { $_.Replace('mdt-01', 'replacethis') }
        Set-Content -Path $FilePath -Value $NewContent
      }
      if ($Content -match 'MDT-01') {
        $Content = $Content | % { $_.Replace('MDT-01', 'replacethis') }
      Set-Content -Path $FilePath -Value $Content
      }
    }
    
  • #16799
    Profile photo of Mathieu Buisson
    Mathieu Buisson
    Participant

    I modified your script to make it easier for me to test and troubleshoot :

    $TextFiles = @('C:\Test.txt',
    			'C:\Test2.txt'
    		)
    foreach ($FilePath in $TextFiles) {
      Write-Output "Searching for string in $FilePath"
      $Content = Get-Content $FilePath
      if ($Content -match 'mdt-01') {
        Write-Output "Found mdt-01 match in $FilePath"
        Write-Output "`$Content is : `r`n $Content"
        $NewContent = $Content | % { $_.Replace('mdt-01', 'replacethis') }
        Write-Output "And after the Replace for mdt-01, `$NewContent is now : `r`n $NewContent"
    
        # Set-Content -Path $FilePath -Value $NewContent
      }
      if ($Content -match 'MDT-01') {
        Write-Output "`$Content is : `r`n $Content"
        $Content = $Content | % { $_.Replace('MDT-01', 'replacethis') }
        Write-Output "And after the Replace for MDT01, `$Content is now : `r`n $Content"
    
        # Set-Content -Path $FilePath -Value $Content
      }
    } 

    Here is the ouput :

    [blockquote]Searching for string in C:\Test.txt
    Found mdt-01 match in C:\Test.txt
    $Content is :
    WSUSServer=http://mdt-01:8530 blablabla WSUSServer=http://MDT-01:8530
    And after the Replace for mdt-01, $NewContent is now :
    WSUSServer=http://replacethis:8530 blablabla WSUSServer=http://MDT-01:8530
    $Content is :
    WSUSServer=http://mdt-01:8530 blablabla WSUSServer=http://MDT-01:8530
    And after the Replace for MDT01, $Content is now :
    WSUSServer=http://mdt-01:8530 blablabla WSUSServer=http://replacethis:8530
    Searching for string in C:\Test2.txt
    Found mdt-01 match in C:\Test2.txt
    $Content is :
    WSUSServer=http://mdt-01:8530 blablabla Test2 WSUSServer=http://MDT-01:8530
    And after the Replace for mdt-01, $NewContent is now :
    WSUSServer=http://replacethis:8530 blablabla Test2 WSUSServer=http://MDT-01:8530
    $Content is :
    WSUSServer=http://mdt-01:8530 blablabla Test2 WSUSServer=http://MDT-01:8530
    And after the Replace for MDT01, $Content is now :
    WSUSServer=http://mdt-01:8530 blablabla Test2 WSUSServer=http://replacethis:8530
    [/blockquote]

    So, we can see that the string "mdt-01" is indeed replaced.
    But at the beginning of the second If statement ( the one looking for "MDT-01"), we see that $Content is back to its original value.
    Your first change (the replacement of "mdt-01" by "replacethis") has been discarded.
    This is because you stored this change in the variable $NewContent, and then, at the beginning of the second If statement, you go back to using $Content.

    So, if we stop using $NewContent and work only with $Content, like so :

    $TextFiles = @('C:\Test.txt',
    			'C:\Test2.txt'
    		)
    foreach ($FilePath in $TextFiles) {
      Write-Output "Searching for string in $FilePath"
      $Content = Get-Content $FilePath
      if ($Content -match 'mdt-01') {
        Write-Output "Found mdt-01 match in $FilePath"
        Write-Output "`$Content is : `r`n $Content"
        $Content = $Content | % { $_.Replace('mdt-01', 'replacethis') }
        Write-Output "And after the Replace for mdt-01, `$Content is now : `r`n $Content"
    
        # Set-Content -Path $FilePath -Value $NewContent
      }
      if ($Content -match 'MDT-01') {
        Write-Output "`$Content is : `r`n $Content"
        $Content = $Content | % { $_.Replace('MDT-01', 'replacethis') }
        Write-Output "And after the Replace for MDT01, `$Content is now : `r`n $Content"
    
        # Set-Content -Path $FilePath -Value $Content
      }
    } 

    Now, the first change is kept and both 'mdt-01' and 'MDT-01' are replaced in the final output :

    [blockquote]Searching for string in C:\Test.txt
    Found mdt-01 match in C:\Test.txt
    $Content is :
    WSUSServer=http://mdt-01:8530 blablabla WSUSServer=http://MDT-01:8530
    And after the Replace for mdt-01, $Content is now :
    WSUSServer=http://replacethis:8530 blablabla WSUSServer=http://MDT-01:8530
    $Content is :
    WSUSServer=http://replacethis:8530 blablabla WSUSServer=http://MDT-01:8530
    And after the Replace for MDT01, $Content is now :
    WSUSServer=http://replacethis:8530 blablabla WSUSServer=http://replacethis:8530
    Searching for string in C:\Test2.txt
    Found mdt-01 match in C:\Test2.txt
    $Content is :
    WSUSServer=http://mdt-01:8530 blablabla Test2 WSUSServer=http://MDT-01:8530
    And after the Replace for mdt-01, $Content is now :
    WSUSServer=http://replacethis:8530 blablabla Test2 WSUSServer=http://MDT-01:8530
    $Content is :
    WSUSServer=http://replacethis:8530 blablabla Test2 WSUSServer=http://MDT-01:8530
    And after the Replace for MDT01, $Content is now :
    WSUSServer=http://replacethis:8530 blablabla Test2 WSUSServer=http://replacethis:8530
    [/blockquote]

    • #16804
      Profile photo of Adam Bertram
      Adam Bertram
      Participant

      Thanks for the $NewContent variable hint. Again, I was originally using all $Content but I get so frustrated sometimes I tend to just throw everything I can can think of at it. This was one of those times. I'll revert back to $Content.

  • #16800
    Profile photo of Daniel Krebs
    Daniel Krebs
    Participant

    Adam and Mathieu,

    As far as I know the -match and -replace operators in PowerShell are case-insensitive. Below example works for me using a different set of test files of course.

    $TextFiles = @('D:\MDTProduction\Control\Bootstrap.ini',
    			'D:\MDTProduction\Control\Customsettings.ini',
    			'D:\MDTBuildLab\Control\Bootstrap.ini',
    			'D:\MDTBuildLab\Control\Customsettings.ini',
    			'D:\MDTBuildLab\Scripts\Realocate.cmd'
    		)
    
    foreach ($FilePath in $TextFiles) {
      $Content = Get-Content $FilePath
    
      Write-Verbose "Searching for string in $FilePath"
      if ($Content -match 'mdt-01') {
        Write-Verbose "Found mdt-01 match in $FilePath"
        $Content = $Content -replace 'mdt-01', 'replacethis'
    
        Set-Content -Path $FilePath -Value $Content
      }
    }
    
    • #16801
      Profile photo of Mathieu Buisson
      Mathieu Buisson
      Participant

      Nice !
      Unlike the Replace method of string objects (used in the original script), the -replace operator is case-insensitive.
      This makes your solution simpler and shorter.

  • #16803
    Profile photo of Adam Bertram
    Adam Bertram
    Participant

    I had tried used the -replace operator but that was probably 10 iterations of me trying ago. I'll try that again. However, I can tell you that -match is case-sensitive. It uses regex.

  • #16809
    Profile photo of Daniel Krebs
    Daniel Krebs
    Participant

    I've done some testing and reading http://technet.microsoft.com/en-us/library/hh847759.aspx. The -match operator is case-insensitive as well. You can use the explicit case-insensitive operator -imatch or the explicit case-sensitive operator if you want.

    $value = "WDSServer=mdt-01"
    
    # Successful match using the default operator
    $value -match 'MDT-01'
    True
    
    # Successful match using the explicit case-insensitive operator
    $value -imatch 'MDT-01'
    True
    
    # No match using the explicit case-sensitive operator
    $value -cmatch 'MDT-01'
    False
    
  • #16810
    Profile photo of Vern Anderson
    Vern Anderson
    Participant

    In my experiences you have to write to the file every single time you replace a value. The only way to avoid doing this is to create a file stream object that allows you to open the file for writing.

    I also have never had any problems with the ".Replace" method being case sensitive.

    NewContent = $Content | % { $_.Replace('mdt-01', 'replacethis')  | Out-File $FilePath -Force -Append}

    [b]Using FileStream instead[/b]
    [url]http://blogs.technet.com/b/heyscriptingguy/archive/2005/02/08/how-can-i-find-and-replace-text-in-a-text-file.aspx[/url]

    -VERN

  • #16811
    Profile photo of Vern Anderson
    Vern Anderson
    Participant

    Also found this blog post about doing it with Set-Content

    [url]http://blogs.technet.com/b/heyscriptingguy/archive/2008/01/17/how-can-i-use-windows-powershell-to-replace-characters-in-a-text-file.aspx[/url]

    There were far more examples using Set-Content so I've been doing it the hard way myself. I plan to revisit my script.

    -VERN

  • #16812
    Profile photo of Adam Bertram
    Adam Bertram
    Participant

    I really appreciate everyone's help. You guys got me over the hump! It ended up being a combination of 2 things; the case sensitivity issue and the fact that this is being run from a WinForm. What works great for me now is:

    $Content = $Content -ireplace 'mdt-01', 'COMPUTERNAME'
    Start-Job -ScriptBlock { Set-Content -Path $Args[0] -Value $Args[1] } -ArgumentList $FilePath, $Content

    I was able to remove both of the if/then blocks by simply using the case-insensitive -ireplace and by using a job (as Don Jones mentioned in my last thread) I was able to successfully replace the text in each text file.

    Thanks for everyone's help!

You must be logged in to reply to this topic.