Tagged: array large text file
September 8, 2017 at 8:48 pm #79168
I have a large text file,over 6 million lines (A list of primes) and I am trying to import it into a array. Get-Content is way to slow so I have tried to use the DotNet to help out. This is what I have tried so far.
$a = New-Object -ComObject Scripting.FileSystemObject $f = $a.OpenTextFile("$ENV:USERPROFILE\Documents\Primes\PrimeList.Cfg", 1) [Array]$Primes=$f.ReadAll()
What I end up with this every element of the array is a single character. Part of the array looks like this.
What I want is
Any help would be appreciated.
September 9, 2017 at 4:17 am #79184
We would need to understand your input file.
I created a file that contains the following
1 2 3 5 7 11 13
I then ran you code against it
$a = New-Object -ComObject Scripting.FileSystemObject $f = $a.OpenTextFile("D:\input.txt", 1) [Array]$Primes=$f.ReadAll() $Primes
PS C:\Windows\System32\WindowsPowerShell\v1.0> $a = New-Object -ComObject Scripting.FileSystemObject $f = $a.OpenTextFile("D:\input.txt", 1) [Array]$Primes=$f.ReadAll() $Primes 1 2 3 5 7 11 13
September 9, 2017 at 4:21 am #79186
I did the same thing, Curtis, and got the same results you did. TeeStar, what do the contents of your file look like? Can you show us maybe the first 10 lines or so? You might open your .cfg file in something like notepad++ and see if there are any special characters that could be causing issues.
September 9, 2017 at 12:41 pm #79199
There must be \r on each line of .cfg file you are trying to read.
I'm very new to Powershell myself but it would happen in Python.
Try to strip any \r and \n character.
September 12, 2017 at 5:47 am #79379
My Notepad ++ output looks fine as well as Notepad. No special characters or anything.
I have tried this
and it seems to work. I don't know why this one and the other does not work, must be the way I saved it perhaps??
September 12, 2017 at 8:13 am #79384
Format-Hexfor looking into your file.
You can see any of "0D/0A" combinations, but only several of it correct 🙂
You must be logged in to reply to this topic.