For processing a huge file, I’d like to only import the third column. I don’t want to read the entire file into memory first and pipe it to Select name
Is there a way for me to specify the index of the column(s) I want? For instance, if I wanted the second column, I’d tell it that I wanted index 1.
If not, what’s the best way to deal with large files that are so big that you run out of RAM reading them in? Is the best method Get-Content | ConvertFrom-CSV? I’m only against that method because of how long it takes. Compared to a simple Import-CSV, it takes at least twice as long. But in cases where Import-CSV can’t be used, is it the best option?
I realize you’re wanting to avoid reading the entire file and pipe to Select-Object, but have you considered using .NET directly with one of the [System.IO.File]:Read* static methods to speed up the read without the overhead of cmdlets? I’ve seen a similar solution in C# and perhaps you may need to look in that direction.