Author Posts

February 25, 2018 at 5:26 pm

Dear Team,

I need to run the below powershell script to run in multiple threads as it needs to check some 40000 + files to find the email address and output the file name and the email address two a output file.

Currently the below script is giving 30 outputs in an hour (sigh) if i can make it run multiple threads may be 50-100 threads at a time i would be able to complete the process much faster. Need your help to achieve this ?

===============================================================================================================================

foreach($user in $users){""|select @{label="proxyAddress2";expression={sls $user C:\Users\admin\Desktop\folder1\*.txt | select -u -exp filename}},@{label="ews";expression={$user}}}

===============================================================================================================================

February 25, 2018 at 8:17 pm

Why are you not using PowerShell workflows / jobs / Runspaces for this. This type of thing is why they exist.

There are many articles on the topic, and a quick search using your favorite engine, and your post title, would bring you back many results / samples / scripts to download and tweak as needed.

Parallel processing with PowerShell

Working in parallel

Whichever approach you end up taking you will be getting PowerShell to run tasks in parallel. That will often require you to have additional instances of PowerShell running. The resources on your admin machine – CPU, memory and network bandwidth – are finite. Keep those in mind so you don't overload the machine and end up getting nothing back.

'blogs.technet.microsoft.com/uktechnet/2016/06/20/parallel-processing-with-powershell'

Invoke-Parallel – Parallel PowerShell

This function will take in a script or scriptblock, and run it against specified objects(s) in parallel. It uses runspaces, as there are many situations where jobs or PSRemoting are not appropriate.

'gallery.technet.microsoft.com/scriptcenter/Run-Parallel-Parallel-377fd430'

Scaling and Queuing PowerShell Background Jobs

The scenario is familiar: You have a file containing a bunch of input that you want to process and you don't want to overburden your computer by starting up hundreds of instances of PowerShell at once to process them.

'blogs.msdn.microsoft.com/powershell/2011/04/04/scaling-and-queuing-powershell-background-jobs'

Multi threading – Powershell to Run Commands in Parallel

Reduce remote powershell execution time using multi threading in powershell v2 We can use multi threading in powershell v2 using start-job command which will execute the script block on multiple servers parallely rather than going one by one. In powershell v3 there is additional

'gallery.technet.microsoft.com/scriptcenter/Multi-threading-Powershell-d2c0e2e5'

Using Background Runspaces Instead of PSJobs For Better Performance

find myself using background runspaces more lately because you do not have to worry about another process (or processes), no serialization of data is required and you can specify throttling of the runspaces via the runspacepool. Another thing I like is the ability to share variables across runspaces.

'learn-powershell.net/2012/05/13/using-background-runspaces-instead-of-psjobs-for-better-performance'

Multithreading with Jobs in PowerShell

NOTE: I have written a better script for generic multithreading which I have covered in my post HERE. If you are looking for a script to cover your every day needs, please read that article instead as I believe it is a better script. This script, however, is easier to understand if you are looking to learn this for yourself!

'get-blog.com/?p=22'