Piping or Export-Csv limit?

Welcome Forums General PowerShell Q&A Piping or Export-Csv limit?

This topic contains 9 replies, has 4 voices, and was last updated by

 
Participant
1 week, 5 days ago.

  • Author
    Posts
  • #180830

    Participant
    Topics: 2
    Replies: 9
    Points: 3
    Rank: Member

    I have a bit of code that ends in an export-csv that is giving me blank files.

    $output | export-csv -Path 'c:\users\me\desktop\output.csv -NoTypeInformation

    $output when output to the screen is loaded with data I would expect to see

    $output.count says its 17102 items

    $output.capacity states 32768

    $output | out-gridview gives the same results as the export-csv, 17k blank lines

    PSVersion: 5.1

  • #180839
    js

    Participant
    Topics: 25
    Replies: 692
    Points: 1,721
    Helping Hand
    Rank: Community Hero

    Are they all the same type of object?

    • #180857

      Participant
      Topics: 2
      Replies: 9
      Points: 3
      Rank: Member

      Yes they are. Here's the full script:

      $namespaces = Import-Csv 'C:\Users\me\Desktop\namespaces.csv'
      $FolderCollection = New-Object System.Collections.Generic.List[System.Object]
      
      foreach($namespace in $namespaces){
      $childshare = Get-ChildItem -Directory -Path $namespace.Name #-Recurse -Depth 3 -Force
      For($i = 1; $i -le $childshare.Count; $i++){
      $sharecount = $childshare.Count
      Write-Progress -Activity "Gathering Permissions $i of $sharecount" -PercentComplete ($i/$childshare.Count*100)
      foreach($share in $childshare){
      $acl = Get-Acl -Path $share.FullName
      foreach($access in $acl.Access){
      $secgroup = ($access.IdentityReference).ToString()
      $secgroup = $secgroup.Substring(10)
      $ObjProp = [ordered]@{'This Share'=$share.FullName;'Grants these rights'=$Access.FileSystemRights;'To these groups'=$Access.IdentityReference;'Which consists of these members'=(get-adgroupmember $($access.IdentityReference.ToString().split('\')[1]) -Recursive -ErrorAction SilentlyContinue | select -ExpandProperty name ) -join '; '}
      $CollectionObject = New-Object -TypeName PSObject -Property $ObjProp
      $FolderCollection.add($CollectionObject) 
      }
      }
      }
      }
      
      $FolderCollection | export-csv -Path C:\Users\me\Desktop\allDFS_permissions.csv - -NoTypeInformation
    • #180860
      js

      Participant
      Topics: 25
      Replies: 692
      Points: 1,721
      Helping Hand
      Rank: Community Hero

      What's an example of one object that works and one object that exports to a blank line?

    • #181404

      Participant
      Topics: 2
      Replies: 9
      Points: 3
      Rank: Member

      In this version of the script, nothing exports, everything is blank. I have another script that does something similar and works just fine. That code is:

      $allshares = Import-Csv "C:\users\me\Desktop\allshares.csv"
      $FolderCollection = New-Object System.Collections.Generic.List[System.Object]
      foreach($share in $allshares){
      $Acl = Get-Acl -Path $share.Name
      
      
      foreach($access in $acl.Access){
      $ObjProp = [ordered]@{'This Share'=$share.Name;'Grants these rights'=$Access.FileSystemRights;'To these groups'=$Access.IdentityReference;'Which consists of these members'=(get-adgroupmember $($access.IdentityReference.ToString().split('\')[1]) -Recursive | select -ExpandProperty name) -join '; '}
      $CollectionObject = New-Object -TypeName PSObject -Property $ObjProp
      $FolderCollection.add($CollectionObject)
      }
      }
      $FolderCollection | export-csv -Path "c:\users\me\desktop\nonDFSpermissions.csv" -NoTypeInformation
    • #181695

      Participant
      Topics: 2
      Replies: 9
      Points: 3
      Rank: Member

      Any more insight? I'm really at a loss as to what is going on here.

  • #180854

    Participant
    Topics: 6
    Replies: 97
    Points: 239
    Helping Hand
    Rank: Participant

    It sounds like $output isn't an object with multiple attributes, etc. Can you give us an example of the output you see at the console and maybe how you're creating the $output variable.

    • #181407

      Participant
      Topics: 2
      Replies: 9
      Points: 3
      Rank: Member
      PS C:\Windows\system32> $FolderCollection
      
      
      This Share Grants these rights To these groups Which consists of these members 
      ---------- ------------------- --------------- ------------------------------- 
      \\domain.local\appdata\Citrix Modify, Synchronize domain\FileChange P*-A**** P****; P*-D**** E****; P*-D**** V******; P*-Q***** N****; P*-M*** M****; P*...
      
      
    • #181692

      Participant
      Topics: 2
      Replies: 9
      Points: 3
      Rank: Member

      Any more insight? I've tried converting the list to an array but still getting blank output. I'm all out of ideas...

  • #180866

    Participant
    Topics: 2
    Replies: 497
    Points: 1,230
    Helping Hand
    Rank: Community Hero

    Judging by your code sample, $Output is a generic List collection.

    For these collections, Count is a count of the objects contained in the List currently. Capacity is the currently allocated space for more objects to be added before it has to re-allocate a new collection internally to store more items.

    Essentially, List is a front-end that keeps an array hidden internally with (often) more capacity than it really strictly needs to have, because by design it's expected that you'll be adding or removing items from the list, and having a bit of additional capacity is helpful to avoid expensive operations where it has to re-allocate an entire new array because it runs out of space.

    So you can think of Count as the size of the List, but Capacity is the size of the currently-allocated internal array where items are actually stored. Once Count reaches the Capacity value, the List will automatically re-allocate a new array the next time you .Add() to the List — if I recall correctly, the new array it allocates is typically double the size of the one whose capacity you've just exceeded. So, when adding many items to the list, you'll see its Capacity change as it allocates new arrays whenever it's needed.

    If you already know the approximate size of your List (within, say, an order of magnitude or thereabouts), you can actually specify its initial Capacity by either:

    1. Creating the list with a preset initial capacity with [System.Collections.Generic.List[object]]::new($Capacity) where $Capacity is an integer.
    2. Creating the list by passing in an existing array for it to copy. It will have an initial capacity equal to the size of the array, if I recall correctly: [System.Collections.Generic.List[object]]::new($array)

    So the fact that it has additional Capacity above and beyond your actual Count of objects is simple an implementation detail; you don't have additional items hiding in your List that aren't being exported.

    Your problem with blank exported files is coming from somewhere else. 🙂

You must be logged in to reply to this topic.