Why does Powershell output unix text? (really utf-16)

Welcome Forums General PowerShell Q&A Why does Powershell output unix text? (really utf-16)

This topic contains 4 replies, has 3 voices, and was last updated by

js
 
Participant
3 months, 2 weeks ago.

  • Author
    Posts
  • #110146
    js

    Participant
    Points: 319
    Helping Hand
    Rank: Contributor

    I happened to notice that when saving output with ">" or out-file, the result is a unix text file with no carriage returns, just "`n", not "`r`n" (EDIT: I was wrong). And Infoblox happens to not like unix text files for csv importing. I know how to work around it, but I found it surprising.

  • #110149
    Jon

    Participant
    Points: 37
    Rank: Member

    https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/out-file?view=powershell-5.1

    Encoding
    Specifies the type of character encoding used in the file. The acceptable values for this parameter are:

    • Unknown
    • String
    • Unicode
    • BigEndianUnicode
    • UTF8
    • UTF7
    • UTF32
    • ASCII
    • Default
    • OEM

    Unicode is the default.
    "Default" uses the encoding of the system's current ANSI code page.
    "OEM" uses the current original equipment manufacturer code page identifier for the operating system.

     

    If you are trying to export a csv, why wouldn't you use export-csv vs out-file?

  • #110158
    js

    Participant
    Points: 319
    Helping Hand
    Rank: Contributor

    Yes, export-csv outputs windows text, or "ascii". It depends on the situation. Sometimes I don't have an object to being with. The default encoding seems to be "Unicode UTF-16 Little-Endian" for out-file and ">", according to "Get-FileEncoding" (from https://gist.github.com/jpoehls/). oem and default seem to be the same as ascii. Default isn't the default.

  • #110159

    Participant
    Points: 290
    Helping Hand
    Rank: Contributor

    If you're doing anything more than the simplest of text output, I would not recommend using the redirection operators to output to files. They're simplistic and are good for quick short use, not meant for anything even remotely complicated.

    Instead, pipe your output into Add- or Set-Content, which will allow you to properly handle encoding and most other things you'll need without issue.

  • #110171
    js

    Participant
    Points: 319
    Helping Hand
    Rank: Contributor

    To me, this is just another bug or sub-optimal feature that's changed in PS 6.

    EDIT: PS 6 uses UTF8-NO-BOM, which looks a lot like ANSI/ASCII: https://docs.microsoft.com/en-us/powershell/scripting/whats-new/what-s-new-in-powershell-core-60?view=powershell-6#default-encoding-is-utf-8-without-a-bom-except-for-new-modulemanifest

    EDIT: Yes, the carriage returns don't matter, it's the ansi/ascii/utf-8 encoding. Some sites say unicode isn't an encoding, it's a character set. 5.1 Out-File doesn't even make unix text. It makes utf-16.

The topic ‘Why does Powershell output unix text? (really utf-16)’ is closed to new replies.